Counting Index Entries

To see the number of entries in a given index, you can use the COUNT() function.

Syntax:

SELECT COUNT(*) AS size FROM INDEX:<index-name>

Example:

• Counting the entries on the index dictionary:

  orientdb> SELECT COUNT(*) AS size FROM INDEX:dictionary
  

Querying Keys from Indexes

You can query all keys in an index using the SELECT command.

Syntax:

SELECT key FROM INDEX:<index-name>

Example:

• Querying the keys in the index dictionary:

  orientdb> SELECT key FROM INDEX:dictionary
  

Querying Index Entries

You can query for all entries on an index as key and rid pairs.

Syntax:

SELECT key, value FROM INDEX:<index-name>

Example:

• Querying the key/rid pairs from the index dictionary:

  orientdb> SELECT key, value FROM INDEX:dictionary
  

Clearing Indexes

Remove all entries from an index. After running this command, the index is empty.

Syntax:

DELETE FROM INDEX:<index-name>

Example:

• Removing all entries from the index dictionary:

  orientdb> DELETE FROM INDEX:dictionary
  

Query the available indexes

To access to the indexes, you can use SQL.

Create your index engine

Here you can find a guide how to create a custom index engine.

SB-Tree Index Algorithm

This indexing algorithm provides a good mix of features, similar to the features available from other index types. It is good for general use and is durable, transactional and supports range queries. There are four index types that utilize the SB-Tree index algorithm:

• UNIQUE Does not allow duplicate keys, fails when it encounters duplicates.
• NOTUNIQUE Does allow duplicate keys.
• FULLTEXT Indexes to any single word of text.
• DICTIONARY Does not allow duplicate keys, overwrites when it encounters duplicates.

For more information on FULLTEXT_HASH_INDEX, see FullText Index.

The SB-Tree index algorithm is based on the B-Tree index algorithm. It has been adapted with several optimizations, which relate to data insertion and range queries. As is the case with all other tree-based indexes, SB-Tree index algorithm experiences log(N) complexity, but the base to this logarithm is about 500.

NOTE: There is an issue in the replacement of indexes based on B-Tree with those based on COLA Tree to avoid slowdowns introduced by random I/O operations. For more information see Issue #1756.

Hash Index Algorithm

This indexing algorithm provides a fast lookup and is very light on disk usage. It is durable and transactional, but does not support range queries. It is similar to a HashMap, which makes it faster on punctual lookups and it consumes less resources than other index types. The Hash index algorithm supports four index types, which have been available since version 1.5.x:

• UNIQUE_HASH_INDEX Does not allow duplicate keys, it fails when it encounters duplicates.
• NOTUNIQUE_HASH_INDEX Does allow duplicate keys.
• FULLTEXT_HASH_INDEX Indexes to any single word.
• DICTIONARY Does not allow duplicate keys, it overwrites when it encounters duplicates.

For more information on FULLTEXT_HASH_INDEX, see FullText Index.

Hash indexes are able to perform index read operations in one I/O operation and write operations in a maximum of three I/O operations. The Hash Index algorithm is based on the Extendible Hashing algorithm. Despite not providing support for range queries, it is noticeably faster than SB-Tree Index Algorithms, (about twice as fast when querying through ten million records).

NOTE: There is an issue relating to the enhancement of Hash indexes to avoid slowdowns introduced by random I/O operations using LSM Tree approaches. For more information, see Issue #1757.

Auto Sharding Index Algorithm

(Since v2.2)

This indexing algorithm is based on the DHT concept, where they keys are stored on different partition, based on the Murmur3 hash function.

Auto Sharding Index supports the following index types:

• UNIQUE_HASH_INDEX Does not allow duplicate keys, it fails when it encounters duplicates.
• NOTUNIQUE_HASH_INDEX Does allow duplicate keys.

Under the hood, this index creates multiple Hash Indexes, one per cluster. So if you have 8 clusters for the class "Employee", this index will create, at the beginning, 8 Hash Indexes.

Since this index is based on the Hash Index, it's able to perform index read operations in one I/O operation and write operations in a maximum of three I/O operations. The Hash Index algorithm is based on the Extendible Hashing algorithm. Despite not providing support for range queries, it is noticeably faster than SB-Tree Index Algorithms, (about twice as fast when querying through ten million records).

Usage

Create an index by passing "AUTOSHARDING" as index engine:

final OClass cls = db.createClass("Log");
cls.createProperty("key", OType.LONG);
cls.createIndex("idx_LogKey", OClass.INDEX_TYPE.UNIQUE.toString(),
    (OProgressListener) null, (ODocument) null, "AUTOSHARDING", new String[] { "key" });

Performance

On multi-core hw, using this index instead of Hash Index gives about +50% more throughput on insertion on a 8 cores machine.

Distributed

The fully distributed version of this index will be supported in v3.0. In v2.2 each node has own copy of the index with all the partitions.

Internals

This is the algorithm for the put(key,value):

int partition = Murmur3_hash(key) % partitions;
getSubIndex(partition).put(key,value);

This is for the value = get(key):

int partition = Murmur3_hash(key) % partitions;
return getSubIndex(partition).get(key);

FullText Indexes

The SB-Tree index algorithm provides support for FullText indexes. These indexes allow you to index text as a single word and its radix. FullText indexes are like having a search engine on your database.

NOTE: Bear in mind that there is a difference between FULLTEXT without the LUCENE operator, which uses a FullText index with the SB-Tree index algorithm and FULLTEXT with the LUCENE operator, which uses a FullText index through the Lucene Engine.

For more information on the latter, see Lucene FullText Index.

Creating FullText Indexes

If you want to create an index using the FullText SB-Tree index algorithm, you can do so using the CREATE INDEX command.

orientdb> CREATE INDEX City.name ON City(name) FULLTEXT

This creates a FullText index on the property name of the class City, using the default configuration.

FullText Index Parameters

In the event that the default FullText Index configuration is not sufficient to your needs, there are a number of parameters available to fine tune how it generates the index.

To configure a FullText Index, from version 1.7 on, you can do so through the OrientDB console or the Java API. When configuring the index from the console, use the CREATE INDEX command with the METADATA operator.

orientdb> CREATE INDEX City.name ON City(name) FULLTEXT METADATA 
          {"indexRadix": true, "ignoreChars": "&", "separatorChars": " |()", 
          "minWordLength": 4, "stopWords": ["the", "of"]}

Alternatively, you can configure the index in Java.

OClass city = db.getClass("City");
ODocument metadata = new ODocument();
metadata.field("indexRadix", true);
metadata.field("stopWords", Arrays.asList(new String[] { "the", "in", "a", "at" }));
metadata.field("separatorChars", " :;?[](.md)");
metadata.field("ignoreChars", "$&");
metadata.field("minWordLength", 5);
city.createIndex("City.name", "FULLTEXT", null, metadata, null, new String[] { "name" });

Lucene FullText Index

In addition to the standard FullText Index, which uses the SB-Tree index algorithm, you can also create FullText indexes using the Lucene Engine . Apache LuceneTM is a high-performance, full-featured text search engine library written entirely in Java. Check the Lucene documentation for a full overview of its capabilities.

How Lucene's works?

Let's look at a sample corpus of five documents:

• My sister is coming for the holidays.
• The holidays are a chance for family meeting.
• Who did your sister meet?
• It takes an hour to make fudge.
• My sister makes awesome fudge.

What does Lucene do? Lucene is a full text search library. Search has two principal stages: indexing and retrieval.

During indexing, each document is broken into words, and the list of documents containing each word is stored in a list called the postings list. The posting list for the word my is:

my --> 1,5

Posting list for others terms:

fudge --> 4,5

sister --> 1,2,3,5

fudge --> 4,5

The index consists of all the posting lists for the words in the corpus. Indexing must be done before retrieval, and we can only retrieve documents that were indexed.

Retrieval is the process starting with a query and ending with a ranked list of documents. Say the query is "my fudge". In order to find matches for the query, we break it into the individual words, and go to the posting lists. The full list of documents containing the keywords is [1,4,5]. Note that the query is broken into words (terms) and each term is matched with the terms in the index. Lucene's default operator is OR, so it retrieves the documents tha contain my OR fudge. If we want to retrieve documents that contain both my and fudge, rewrite the query: "+my +fudge".

Lucene doesn't work as a LIKE operator on steroids, it works on single terms. Terms are produced analyzing the provided text, so the right analyzer should be configured. On the other side, it offers a complete query language, well documented here:

Index creation

To create an index based on Lucene

CREATE INDEX  ON  (prop-names) FULLTEXT ENGINE LUCENE [{json metadata}]

The following SQL statement will create a FullText index on the property name for the class City, using the Lucene Engine.

CREATE INDEX City.name ON City(name) FULLTEXT ENGINE LUCENE

Indexes can also be created on n-properties. For example, create an index on the properties name and description on the class City.

CREATE INDEX City.name_description ON City(name, description)
          FULLTEXT ENGINE LUCENE

When multiple properties should be indexed, define a single multi-field index over the class. A single multi-field index needs less resources, such as file handlers. Moreover, it is easy to write better Lucene queries. The default analyzer used by OrientDB when a Lucene index is created is the StandardAnalyzer. The StandardAnalyzer usually works fine with western languages, but Lucene offers analyzer for different languages and use cases.

Two minutes tutorial

Open studio or console and create a sample dataset:

CREATE CLASS Item;
CREATE PROPERTY Item.text STRING;
CREATE INDEX Item.text ON Item(text) FULLTEXT ENGINE LUCENE;
INSERT INTO Item (text) VALUES ('My sister is coming for the holidays.');
INSERT INTO Item (text) VALUES ('The holidays are a chance for family meeting.');
INSERT INTO Item (text) VALUES ('Who did your sister meet?');
INSERT INTO Item (text) VALUES ('It takes an hour to make fudge.');
INSERT INTO Item (text) VALUES ('My sister makes awesome fudge.');

Search all documents that contain sister:

SELECT FROM Item WHERE SEARCH_CLASS("sister") = true

Search all documents that contain sister AND coming:

SELECT FROM Item WHERE SEARCH_CLASS("+sister +coming") = true

Search all documents that contain sister but NOT coming:

SELECT FROM Item WHERE SEARCH_CLASS("+sister -coming") = true

Search all documents that contain the phrase sister meet:

SELECT FROM Item WHERE SEARCH_CLASS(' "sister meet" ') = true

Search all documents that contain terms starting with meet:

SELECT FROM Item WHERE SEARCH_CLASS('meet*') = true

To better understand how the query parser work, read carefully the official documentation and play with the above documents.

Customize Analyzers

In addition to the StandardAnalyzer, full text indexes can be configured to use different analyzer by the METADATA operator through CREATE INDEX.

Configure the index on City.name to use the EnglishAnalyzer:

CREATE INDEX City.name ON City(name)
            FULLTEXT ENGINE LUCENE METADATA {
                "analyzer": "org.apache.lucene.analysis.en.EnglishAnalyzer"
            }

Configure the index on City.name to use different analyzers for indexing and querying.

CREATE INDEX City.name ON City(name)
            FULLTEXT ENGINE LUCENE METADATA {
                "index": "org.apache.lucene.analysis.en.EnglishAnalyzer",
                "query": "org.apache.lucene.analysis.standard.StandardAnalyzer"
          }

EnglishAnalyzer will be used to analyze text while indexing and the StandardAnalyzer will be used to analyze query text.

A very detailed configuration, on multi-field index configuration, could be:

CREATE INDEX Song.fulltext ON Song(name, lyrics, title, author, description)
            FULLTEXT ENGINE LUCENE METADATA {
                "default": "org.apache.lucene.analysis.standard.StandardAnalyzer",
                "index": "org.apache.lucene.analysis.core.KeywordAnalyzer",
                "query": "org.apache.lucene.analysis.standard.StandardAnalyzer",
                "name_index": "org.apache.lucene.analysis.standard.StandardAnalyzer",
                "name_query": "org.apache.lucene.analysis.core.KeywordAnalyzer",
                "lyrics_index": "org.apache.lucene.analysis.en.EnglishAnalyzer",
                "title_index": "org.apache.lucene.analysis.en.EnglishAnalyzer",
                "title_query": "org.apache.lucene.analysis.en.EnglishAnalyzer",
                "author_query": "org.apache.lucene.analysis.core.KeywordAnalyzer",
                "description_index": "org.apache.lucene.analysis.standard.StandardAnalyzer",
                "description_index_stopwords": [
                  "the",
                  "is"
                ]
            }

With this configuration, the underlying Lucene index will works in different way on each field:

• name: indexed with StandardAnalyzer, searched with KeywordAnalyzer (it's a strange choice, but possible)
• lyrics: indexed with EnglishAnalyzer, searched with default query analyzer StandardAnalyzer
• title: indexed and searched with EnglishAnalyzer
• author: indexed and searched with KeywordAnalyzer
• description: indexed with StandardAnalyzer with a given set of stop-words that overrides the internal set

Analysis is the foundation of Lucene. By default the StandardAnalyzer removes english stop-words and punctuation and lowercase the generated terms:

The holidays are a chance for family meeting!

Would produce

• holidays
• are
• chance
• for
• family
• meeting

Each analyzer has its set of stop-words and tokenize the text in a different way. Read the full (documentation)[http://lucene.apache.org/core/6_6_0/].

Query parser

It is possible to configure some behavior of the Lucene query parser Query parser's behavior can be configured at index creation time and overridden at runtime.

Allow Leading Wildcard

Lucene by default doesn't support leading wildcard: Lucene wildcard support

It is possible to override this behavior with a dedicated flag on meta-data:

{
  "allowLeadingWildcard": true
}

CREATE INDEX City.name ON City(name)
            FULLTEXT ENGINE LUCENE METADATA {
                "allowLeadingWildcard": true
            }

Use this flag carefully, as stated in the Lucene FAQ:

Note that this can be an expensive operation: it requires scanning the list of tokens in the index in its entirety to look for those that match the pattern.

Disable lower case on terms

Lucene's QueryParser applies a lower case filter on expanded queries by default. It is possible to override this behavior with a dedicated flag on meta-data:

{
  "lowercaseExpandedTerms": false
}

It is useful when used in pair with keyword analyzer:

CREATE INDEX City.name ON City(name)
            FULLTEXT ENGINE LUCENE METADATA {
              "lowercaseExpandedTerms": false,
              "default" : "org.apache.lucene.analysis.core.KeywordAnalyzer"
            }

With lowercaseExpandedTerms set to false, these two queries will return different results:

SELECT from Person WHERE SEARCH_CLASS("NAME") = true

SELECT from Person WHERE WHERE SEARCH_CLASS("name") = true

Querying Lucene FullText Indexes

OrientDB 3.0.x introduced search functions: SEARCH_CLASS, SEARCH_FIELDS, SEARCH_INDEX, SEARCH_MORE Every function accepts as last, optional, parameter a JSON with additional configuration.

SEARCH_CLASS

The best way to use the search capabilities of OrientDB is to define a single multi-fields index and use the SEARCH_CLASS function. In case more than one full-text index are defined over a class, an error is raised in case of SEARCH_CLASS invocation.

Suppose to have this index

CREATE INDEX City.fulltext ON City(name, description) FULLTEXT ENGINE LUCENE 

A query that retrieve cities with the name starting with cas and description containing the word beautiful:

SELECT FROM City WHERE SEARCH_CLASS("+name:cas*  +description:beautiful") = true

The function accepts metadata JSON as second parameter:

SELECT FROM City WHERE SEARCH_CLASS("+name:cas*  +description:beautiful", {
    "allowLeadingWildcard": true ,
    "lowercaseExpandedTerms": false,
    "boost": {
        "name": 2
    },
    "highlight": {
        "fields": ["name"],
        "start": "",
        "end": ""
    }
}) = true

The query shows query parser's configuration overrides, boost of field name with highlight. Highlight and boost will be explained later.

SEARCH_MORE

OrientDB exposes the Lucene's more like this capability with a dedicated function.

The first parameter is the array of RID of elements to be used to calculate similarity, the second parameter the usual metadata JSON used to tune the query behaviour.

SELECT FROM City WHERE SEARCH_MORE([#25:2, #25:3],{'minTermFreq':1, 'minDocFreq':1} ) = true

It is possible to use a query to gather RID of documents to be used to calculate similarity:

SELECT FROM City
    let $a=(SELECT @rid FROM City WHERE name = 'Rome')
    WHERE SEARCH_MORE( $a, { 'minTermFreq':1, 'minDocFreq':1} ) = true

Lucene's MLT has a lot of parameter, and all these are exposed through the metadata JSON: http://lucene.apache.org/core/6_6_0/queries/org/apache/lucene/queries/mlt/MoreLikeThis.html

• fieldNames: array of field's names to be used to extract content
• maxQueryTerms
• minDocFreq
• maxDocFreq
• minTermFreq
• boost
• boostFactor
• maxWordLen
• minWordLen
• maxNumTokensParsed
• stopWords

Query parser's runtime configuration

It is possible to override the query parser's configuration given at creation index time at runtime passing a json:

SELECT from Person WHERE SEARCH_CLASS("bob",{
        "allowLeadingWildcard": true ,
        "lowercaseExpandedTerms": false
    } ) = true

The same can be done for query analyzer, overriding the configuration given at index creation's time:

SELECT from Person WHERE SEARCH_CLASS("bob",{
        "customAnalysis": true ,
        "query": "org.apache.lucene.analysis.standard.StandardAnalyzer",
        "name_query": "org.apache.lucene.analysis.en.EnglishAnalyzer"
    } ) = true

The customAnalysis flag is mandatory to enable the runtime configuration of query analyzers. The runtime configuration is per query and it isn't stored nor reused for a subsequent query. The custom configuration can be used with all the functions.

SEARCH_INDEX

The SEARCH_INDEX function allows to execute the query on a single index. It is useful if more than one index are defined over a class.

SELECT FROM City WHERE SEARCH_INDEX("City.name", "cas*") = true

The function accepts a JSON as third parameter, as for SEARCH_CLASS.

SEARCH_FIELDS

The SEARCH_FIELDS function allows to execute query over the index that is defined over one or more fields:

SELECT FROM City WHERE SEARCH_FIELDS(["name", "description"], "name:cas* description:beautiful") = true

The function accepts a JSON as third parameter, as for SEARCH_CLASS.

Numeric and date range queries

If the index is defined over a numeric field (INTEGER, LONG, DOUBLE) or a date field (DATE, DATETIME), the engine supports range queries Suppose to have a City class with a multi-field Lucene index defined:

CREATE CLASS CITY EXTENDS V
CREATE PROPERTY CITY.name STRING
CREATE PROPERTY CITY.size INTEGER
CREATE INDEX City.name ON City(name,size) FULLTEXT ENGINE LUCENE

Then query using ranges:

SELECT FROM City WHERE SEARCH_CLASS('name:cas* AND size:[15000 TO 20000]') = true

Ranges can be applied to DATE/DATETIME field as well. Create a Lucene index over a property:

CREATE CLASS Article EXTENDS V
CREATE PROPERTY Article.createdAt DATETIME
CREATE INDEX Article.createdAt  ON Article(createdAt) FULLTEXT ENGINE LUCENE

Then query to retrieve articles published only in a given time range:

SELECT FROM Article WHERE SEARCH_CLASS('[201612221000 TO 201612221100]') =true

Retrieve the Score

When the lucene index is used in a query, the results set carries a context variable for each record representing the score. To display the score add $score in projections.

SELECT *,$score FROM V WHERE name LUCENE "test*"

Highlighting

OrientDB uses the Lucene's highlighter. Highlighting can be configured using the metadata JSON. The highlighted content of a field is returned in a dedicated field suffixed with _hl:

SELECT name, $name_hl, description, $description_hl FROM City
WHERE SEARCH_CLASS("+name:cas*  +description:beautiful", {
    "highlight": {
        "fields": ["name", "description"],
        "start": "",
        "end": ""
    }
}) = true

Parameters

• fields: array of field names to be highlighted
• start: start delimiter for highlighted text (default <B>)
• end: end delimiter for highlighted text (default </B>)
• maxNumFragments: maximum number of text's fragments to highlight

Sorting

Sorting uses SortedDocValuesField which for too large string can cause DocValuesField is too large, must be <= 32766 exception.

In order to disable sorting add the configuration on the metadata ad index creation (From OrientDB 3.0.24).

For all fields:

CREATE INDEX City.name ON City(name)
            FULLTEXT ENGINE LUCENE METADATA {
                "*_index_sorted": false
            }

For single field:

CREATE INDEX City.name ON City(name)
            FULLTEXT ENGINE LUCENE METADATA {
                "name_index_sorted": false
            }

Documents retrieved by a search call are ordered by their score. It is possible to configure the way the document are sorted. Read carefully the official documentation about sorting : https://lucene.apache.org/core/6_6_1/core/org/apache/lucene/search/Sort.html

SELECT name, description, size FROM City
WHERE SEARCH_CLASS("+name:cas*  +description:beautiful", {
    "sort": [ { 'field': 'size', reverse:true, type:'INT' }]
}) = true

Sort over multiple fields is possible:

SELECT name, description, size FROM City
WHERE SEARCH_CLASS("+name:cas*  +description:beautiful", {
    "sort": [
        { 'field': 'size', reverse:true, type:'INT' },
        { 'field': 'name', reverse:false, type:'STRING' },
        { reverse:false, type:'DOC' },
        ]
}) = true

Sort configuration:

• field: is the field name. Could be absent only if the sort type is DOC or INDEX
• reverse: if set to true, will sort for the given field in reverse order
• type: look to https://lucene.apache.org/core/6_6_1/core/org/apache/lucene/search/SortField.Type.html

CUSTOM type is not supported

Cross class search (Enterprise Edition)

Bundled with the enterprise edition there's the SEARCH_CROSS function that is able to search over all the Lucene indexes defined on a database

Suppose to define two indexes:

CREATE INDEX Song.title ON Song (title,author) FULLTEXT ENGINE LUCENE METADATA
CREATE INDEX Author.name on Author(name,score) FULLTEXT ENGINE LUCENE METADATA

Searching for a term on each class implies a lot of different queries to be aggregated.

The SEARCH_CLASS function automatically performs the given query on each full-text index configured inside the database.

SELECT  EXPAND(SEARCH_CROSS('beautiful'))

The query will be execute over all the indexes configured on each field. It is possible to search over a given field of a certain class, just qualify the field names with their class name:

SELECT  EXPAND(SEARCH_CROSS('Song.title:beautiful  Author.name:bob'))

Another way is to use the metadata field _CLASS present in every index:

SELECT expand(SEARCH_CROSS('(+_CLASS:Song +title:beautiful) (+_CLASS:Author +name:bob)') )

All the options of a Lucene's query are allowed: inline boosting, phrase queries, proximity etc.

The function accepts a metadata JSON as second parameter:

SELECT  EXPAND(SEARCH_CROSS('Author.name:bob Song.title:*tain', {"
   "allowLeadingWildcard" : true,
   "boost": {
        "Author.name": 2.0
        }
   }
)

Highlight isn't supported yet.

Lucene Writer fine tuning (expert)

It is possible to fine tune the behaviour of the underlying Lucene's IndexWriter

CREATE INDEX City.name ON City(name)
    FULLTEXT ENGINE LUCENE METADATA {
        "directory_type": "nio",
        "use_compound_file": false,
        "ram_buffer_MB": "16",
        "max_buffered_docs": "-1",
        "max_buffered_delete_terms": "-1",
        "ram_per_thread_MB": "1024",
        "default": "org.apache.lucene.analysis.standard.StandardAnalyzer"
    }

• directory_type: configure the access type to the Lucene's index
  • nio (default): the index is opened with NIOFSDirectory
  • mmap: the index is opened with MMapDirectory
  • ram: index will be created in memory with RAMDirectory
• use_compound_file: default is false
• ram_buffer_MB: size of the document's buffer in MB, default value is 16 MB (which means flush when buffered docs consume approximately 16 MB RAM)
• max_buffered_docs: size of the document's buffer in number of docs, disabled by default (because IndexWriter flushes by RAM usage by default)
• max_buffered_delete_terms: disabled by default (because IndexWriter flushes by RAM usage by default).
• ram_per_thread_MB: default value is 1945

For a detailed explanation of config parameters and IndexWriter behaviour

• indexWriterConfig : https://lucene.apache.org/core/6_6_0/core/org/apache/lucene/index/IndexWriterConfig.html
• indexWriter: https://lucene.apache.org/core/6_6_0/core/org/apache/lucene/index/IndexWriter.html

Index lifecycle

Lucene indexes are lazy. If the index is in idle mode, no reads and no writes, it will be closed. Intervals are fully configurable.

• flushIndexInterval: flushing index interval in milliseconds, default to 20000 (10s)
• closeAfterInterval: closing index interval in milliseconds, default to 120000 (12m)
• firstFlushAfter: first flush time in milliseconds, default to 10000 (10s)

To configure the index lifecycle, just pass the parameters in the JSON of metadata:

CREATE INDEX City.name ON City(name) FULLTEXT ENGINE LUCENE METADATA
{
  "flushIndexInterval": 200000,
  "closeAfterInterval": 200000,
  "firstFlushAfter": 20000
}

Create index using the Java API

The FullText Index with the Lucene Engine is configurable through the Java API.

OSchema schema = databaseDocumentTx.getMetadata().getSchema();
    OClass oClass = schema.createClass("Foo");
    oClass.createProperty("name", OType.STRING);
    oClass.createIndex("City.name", "FULLTEXT", null, null, "LUCENE", new String[] { "name"});

The LUCENE operator (deprecated)

NOTE: LUCENE operator is translated to SEARCH_FIELDS function, but it doesn't support the metadata JSON

You can query the Lucene FullText Index using the custom operator LUCENE with the Query Parser Syntax from the Lucene Engine.

SELECT FROM V WHERE name LUCENE "test*"

This query searches for test, tests, tester, and so on from the property name of the class V. The query can use proximity operator ~, the required (+) and prohibit (-) operators, phrase queries, regexp queries:

SELECT FROM Article WHERE content LUCENE "(+graph -rdbms) AND +cloud"

Working with multiple fields (deprecated)

NOTE: define a single Lucene index on the class and use SEARCH_CLASS function

In addition to the standard Lucene query above, you can also query multiple fields. For example,

SELECT FROM Class WHERE [prop1, prop2] LUCENE "query"

In this case, if the word query is a plain string, the engine parses the query using MultiFieldQueryParser on each indexed field.

To execute a more complex query on each field, surround your query with parentheses, which causes the query to address specific fields.

SELECT FROM Article WHERE [content, author] LUCENE "(content:graph AND author:john)"

Here, the engine parses the query using the QueryParser

Creating a Manual Lucene Index (deprecated)

NOTE: avoid manual Lucene index

The Lucene Engine supports index creation without the need for a class.

Syntax:

CREATE INDEX  FULLTEXT ENGINE LUCENE  [] [METADATA {}]

For example, create a manual index using the CREATE INDEX command:

CREATE INDEX Manual FULLTEXT ENGINE LUCENE STRING, STRING

Once you have created the index Manual, you can insert values in index using the INSERT INTO INDEX:... command.

INSERT INTO INDEX:Manual (key, rid) VALUES(['Enrico', 'Rome'], #5:0)

You can then query the index through SELECT...FROM INDEX::

SELECT FROM INDEX:Manual WHERE key LUCENE "Enrico"

Manual indexes could be created programmatically using the Java API

ODocument meta = new ODocument().field("analyzer", StandardAnalyzer.class.getName());
OIndex index = databaseDocumentTx.getMetadata().getIndexManager()
     .createIndex("apiManual", OClass.INDEX_TYPE.FULLTEXT.toString(),
         new OSimpleKeyIndexDefinition(1, OType.STRING, OType.STRING), null, null, meta, OLuceneIndexFactory.LUCENE_ALGORITHM);

Lucene Spatial

(Versions 2.2 and after only, otherwise look at Legacy section)

Since v 3.0, this module is provided in-bundle with the main distribution (Community and Enterprise Editions).

Install

In versions prior to v 3.0 the spatial plugin was a separate component and needed to be installed manually. This is not the case in v 3.0, where the spatial plugin is included in the main distribution, so there is no need to install it.

Geometry Data

OrientDB supports the following Geometry objects :

• Point (OPoint)
• Line (OLine)
• Polygon (OPolygon)
• MultiPoint (OMultiPoint)
• MultiLine (OMultiline)
• MultiPolygon (OMultiPolygon)
• Geometry Collections

OrientDB stores those objects like embedded documents with special classes. The module creates abstract classes that represent each Geometry object type, and those classes can be embedded in user defined classes to provide geospatial information.

Each spatial classes (Geometry Collection excluded) comes with field coordinates that will be used to store the geometry structure. The "coordinates" field of a geometry object is composed of one position (Point), an array of positions (LineString or MultiPoint), an array of arrays of positions (Polygons, MultiLineStrings) or a multidimensional array of positions (MultiPolygon).

Geometry data Example

Restaurants Domain

CREATE class Restaurant
CREATE PROPERTY Restaurant.name STRING
CREATE PROPERTY Restaurant.location EMBEDDED OPoint

To insert restaurants with location

From SQL

INSERT INTO  Restaurant SET name = 'Dar Poeta', location = {"@class": "OPoint","coordinates" : [12.4684635,41.8914114]}

or as an alternative, if you use WKT format you can use the function ST_GeomFromText to create the OrientDB geometry object.

INSERT INTO  Restaurant SET name = 'Dar Poeta', location = St_GeomFromText("POINT (12.4684635 41.8914114)")

From JAVA

ODocument location = new ODocument("OPoint");
location.field("coordinates", Arrays.asList(12.4684635, 41.8914114));

ODocument doc = new ODocument("Restaurant");
doc.field("name","Dar Poeta");
doc.field("location",location);

doc.save();

A spatial index on the location field s defined by

CREATE INDEX Restaurant.location ON Restaurant(location) SPATIAL ENGINE LUCENE"

Functions

OrientDB follows The Open Geospatial Consortium OGC for extending SQL to support spatial data. OrientDB implements a subset of SQL-MM functions with ST prefix (Spatial Type)

ST_AsText

Syntax : ST_AsText(geom)

Example

SELECT ST_AsText({"@class": "OPoint","coordinates" : [12.4684635,41.8914114]})

Result
----------
POINT (12.4684635 41.8914114)

ST_GeomFromText

Syntax : ST_GeomFromText(text)

Example

select ST_GeomFromText("POINT (12.4684635 41.8914114)")

Result
----------------------------------------------------------------------------------
{"@type":"d","@version":0,"@class":"OPoint","coordinates":[12.4684635,41.8914114]}

ST_AsGeoJSON

Syntax : ST_AsGeoJSON(geom)

Example

select ST_AsGeoJSON(ST_GeomFromText("POINT (12.4684635 41.8914114)"))

Result
----------
{"type":"Point","coordinates":[12.468464,41.891411]}

ST_GeomFromGeoJSON

Syntax : ST_GeomFromGeoJSON(getJsonText)

Example

select ST_GeomFromGeoJSON('{"type":"Point","coordinates":[12.4684635,41.8914114]}')

Result
----------------------------------------------------------------------------------
{"@type":"d","@version":0,"@class":"OPoint","coordinates":[12.4684635,41.8914114]}



### ST_Equals

Returns true if geom1 is spatially equal to geom2

Syntax : ST_Equals(geom1,geom2)

Example
```SQL

SELECT ST_Equals(ST_GeomFromText('LINESTRING(0 0, 10 10)'), ST_GeomFromText('LINESTRING(0 0, 5 5, 10 10)'))

Result
-----------
true

ST_Within

Returns true if geom1 is inside geom2

Syntax : ST_Within(geom1,geom2)

This function will use an index if available.

Example

select * from City where  ST_WITHIN(location,'POLYGON ((12.314015 41.8262816, 12.314015 41.963125, 12.6605063 41.963125, 12.6605063 41.8262816, 12.314015 41.8262816))') = true

ST_DWithin

Returns true if the geometries are within the specified distance of one another

Syntax : ST_DWithin(geom1,geom2,distance)

Example

SELECT ST_DWithin(ST_GeomFromText('POLYGON((0 0, 10 0, 10 5, 0 5, 0 0))'), ST_GeomFromText('POLYGON((12 0, 14 0, 14 6, 12 6, 12 0))'), 2.0d) as distance

SELECT from Polygon where ST_DWithin(geometry, ST_GeomFromText('POLYGON((12 0, 14 0, 14 6, 12 6, 12 0))'), 2.0) = true

ST_Contains

Returns true if geom1 contains geom2

Syntax : ST_Contains(geom1,geom2)

This function will use an index if available.

Example

SELECT ST_Contains(ST_Buffer(ST_GeomFromText('POINT(0 0)'),10),ST_GeomFromText('POINT(0 0)'))

Result
----------
true

SELECT ST_Contains(ST_Buffer(ST_GeomFromText('POINT(0 0)'),10),ST_Buffer(ST_GeomFromText('POINT(0 0)'),20))

Result
----------
false

ST_Disjoint

Returns true if geom1 does not spatially intersects geom2

Syntax: St_Disjoint(geom1,geom2)

This function does not use indexes

Example

SELECT ST_Disjoint(ST_GeomFromText('POINT(0 0)'), ST_GeomFromText('LINESTRING ( 2 0, 0 2 )'));

Result
-----------------
true

SELECT ST_Disjoint(ST_GeomFromText('POINT(0 0)'), ST_GeomFromText('LINESTRING ( 0 0, 0 2 )'));

Result
-----------------
false

ST_Intersects

Returns true if geom1 spatially intersects geom2

Syntax: ST_Intersects(geom1,geom2)

Example

SELECT ST_Intersects(ST_GeomFromText('POINT(0 0)'), ST_GeomFromText('LINESTRING ( 2 0, 0 2 )'));

Result
-------------
false

SELECT ST_Intersects(ST_GeomFromText('POINT(0 0)'), ST_GeomFromText('LINESTRING ( 0 0, 0 2 )'));

Result
-------------
true

ST_AsBinary

Returns the Well-Known Binary (WKB) representation of the geometry

Syntax : ST_AsBinary(geometry)

Example

SELECT ST_AsBinary(ST_GeomFromText('POINT(0 0)'))

ST_Envelope

Returns a geometry representing the bounding box of the supplied geometry

Syntax : ST_Envelope(geometry)

Example

SELECT ST_AsText(ST_Envelope(ST_GeomFromText('POINT(1 3)')));

Result
----------
POINT (1 3)

SELECT ST_AsText(ST_Envelope(ST_GeomFromText('LINESTRING(0 0, 1 3)')))

Result
-----------------------------------
POLYGON ((0 0, 0 3, 1 3, 1 0, 0 0))

ST_Buffer

Returns a geometry that represents all points whose distance from this Geometry is less than or equal to distance.

Syntax: ST_Buffer(geometry,distance [,config])

where config is an additional parameter (JSON) that can be use to set:

quadSegs: int -> number of segments used to approximate a quarter circle (defaults to 8).

{ 
  quadSegs : 1
}

endCap : round|flat|square -> endcap style (defaults to "round").

{
  endCap : 'square'
}

join : round|mitre|bevel -> join style (defaults to "round")

{ 
  join : 'bevel'
}

mitre : double -> mitre ratio limit (only affects mitered join style).

{ 
  join : 'mitre', 
  mitre : 5.0
}

Example

SELECT ST_AsText(ST_Buffer(ST_GeomFromText('POINT(100 90)'),50))

SELECT ST_AsText(ST_Buffer(ST_GeomFromText('POINT(100 90)'), 50, { quadSegs : 2 }));

Operators

A && B

Overlaps operator. Returns true if bounding box of A overlaps bounding box of B. This operator will use an index if available.

Example

CREATE CLASS TestLineString
CREATE PROPERTY TestLineString.location EMBEDDED OLineString
INSERT INTO TestLineSTring SET name = 'Test1' , location = St_GeomFromText("LINESTRING(0 0, 3 3)")
INSERT INTO TestLineSTring SET name = 'Test2' , location = St_GeomFromText("LINESTRING(0 1, 0 5)")
SELECT FROM TestLineString WHERE location && "LINESTRING(1 2, 4 6)"

Spatial Indexes

To speed up spatial search and match condition, spatial operators and functions can use a spatial index if defined to avoid sequential full scan of every records.

The current spatial index implementation is built upon lucene-spatial.

The syntax for creating a spatial index on a geometry field is :

CREATE INDEX <name> ON <class-name> (geometry-field) SPATIAL ENGINE LUCENE

Legacy

Before v2.2, OrientDB was able to only index Points. Other Shapes like rectangles and polygons are managed starting from v2.2 (look above). This is the legacy section for databases created before v2.2.

How to create a Spatial Index

The index can be created on a class that has two fields declared as DOUBLE (latitude,longitude) that are the coordinates of the Point.

For example we have a class Place with 2 double fields latitude and longitude. To create the spatial index on Place use this syntax.

CREATE INDEX Place.l_lon ON Place(latitude,longitude) SPATIAL ENGINE LUCENE

The Index can also be created with the Java Api. Example:

OSchema schema = databaseDocumentTx.getMetadata().getSchema();
OClass oClass = schema.createClass("Place");
oClass.createProperty("latitude", OType.DOUBLE);
oClass.createProperty("longitude", OType.DOUBLE);
oClass.createProperty("name", OType.STRING);
oClass.createIndex("Place.latitude_longitude", "SPATIAL", null, null, "LUCENE", new String[] { "latitude", "longitude" });

How to query the Spatial Index

Two custom operators has been added to query the Spatial Index:

1. NEAR: to find all Points near a given location (latitude, longitude)
2. WITHIN: to find all Points that are within a given Shape

NEAR operator

Finds all Points near a given location (latitude, longitude).

Syntax

SELECT FROM Class WHERE [<lat-field>,<long-field>] NEAR [lat,lon]

To specify maxDistance we have to pass a special variable in the context:

SELECT FROM Class WHERE [<lat-field>,<long-field>,$spatial] NEAR [lat,lon,{"maxDistance": distance}]

The maxDistance field has to be in kilometers, not radians. Results are sorted from nearest to farthest.

To know the exact distance between your Point and the Points matched, use the special variable in the context $distance.

SELECT *, $distance FROM Class WHERE [<lat-field>,<long-field>,$spatial] NEAR [lat,lon,{"maxDistance": distance}]

Examples

Let's take the example we have written before. We have a Spatial Index on Class Place on properties latitude and longitude.

Example: How to find the nearest Place of a given point:

SELECT *,$distance FROM Place WHERE [latitude,longitude,$spatial] NEAR [51.507222,-0.1275,{"maxDistance":1}]

WITHIN operator

Finds all Points that are within a given Shape.

	The current release supports only Bounding Box shape

Syntax

SELECT FROM Class WHERE [<lat field>,<long field>] WITHIN [ [ <lat1>, <lon1> ] , [ <lat2>, <lon2> ] ... ]

Examples

Example with previous configuration:

SELECT * FROM Places WHERE [latitude,longitude] WITHIN [[51.507222,-0.1275],[55.507222,-0.1275]]

This query will return all Places within the given Bounding Box.

Managing Dates

OrientDB treats dates as first class citizens. Internally, it saves dates in the Unix time format. Meaning, it stores dates as a long variable, which contains the count in milliseconds since the Unix Epoch, (that is, 1 January 1970).

Date and Datetime Formats

In order to make the internal count from the Unix Epoch into something human readable, OrientDB formats the count into date and datetime formats. By default, these formats are:

• Date Format: yyyy-MM-dd
• Datetime Format: yyyy-MM-dd HH:mm:ss

In the event that these default formats are not sufficient for the needs of your application, you can customize them through ALTER DATABASE...DATEFORMAT and DATETIMEFORMAT commands. For instance,

orientdb> ALTER DATABASE DATEFORMAT "dd MMMM yyyy"

This command updates the current database to use the English format for dates. That is, 14 Febr 2015.

SQL Functions and Methods

To simplify the management of dates, OrientDB SQL automatically parses dates to and from strings and longs. These functions and methods provide you with more control to manage dates:

For example, consider a case where you need to extract only the years for date entries and to arrange them in order. You can use the .format() method to extract dates into different formats.

orientdb> SELECT @RID, id, date.format('yyyy') AS year FROM Order

--------+----+------+
 @RID   | id | year |
--------+----+------+
 #31:10 | 92 | 2015 |
 #31:10 | 44 | 2014 |
 #31:10 | 32 | 2014 |
 #31:10 | 21 | 2013 |
--------+----+------+

In addition to this, you can also group the results. For instance, extracting the number of orders grouped by year.

orientdb> SELECT date.format('yyyy') AS Year, COUNT(*) AS Total 
          FROM Order ORDER BY Year

------+--------+
 Year |  Total |
------+--------+
 2015 |      1 |
 2014 |      2 |
 2013 |      1 |
------+--------+

Dates before 1970

While you may find the default system for managing dates in OrientDB sufficient for your needs, there are some cases where it may not prove so. For instance, consider a database of archaeological finds, a number of which date to periods not only before 1970 but possibly even before the Common Era. You can manage this by defining an era or epoch variable in your dates.

For example, consider an instance where you want to add a record noting the date for the foundation of Rome, which is traditionally referred to as April 21, 753 BC. To enter dates before the Common Era, first run the [ALTER DATABASE DATETIMEFORMAT] command to add the GG variable to use in referencing the epoch.

orientdb> ALTER DATABASE DATETIMEFORMAT "yyyy-MM-dd HH:mm:ss GG"

Once you've run this command, you can create a record that references date and datetime by epoch.

orientdb> CREATE VERTEX V SET city = "Rome", date = DATE("0753-04-21 00:00:00 BC")
orientdb> SELECT @RID, city, date FROM V

-------+------+------------------------+
 @RID  | city | date                   |
-------+------+------------------------+
 #9:10 | Rome | 0753-04-21 00:00:00 BC |
-------+------+------------------------+

Using .format() on Insertion

In addition to the above method, instead of changing the date and datetime formats for the database, you can format the results as you insert the date.

orientdb> CREATE VERTEX V SET city = "Rome", date = DATE("yyyy-MM-dd HH:mm:ss GG")
orientdb> SELECT @RID, city, date FROM V

------+------+------------------------+
 @RID | city | date                   |
------+------+------------------------+
 #9:4 | Rome | 0753-04-21 00:00:00 BC |
------+------+------------------------+

Here, you again create a vertex for the traditional date of the foundation of Rome. However, instead of altering the database, you format the date field in CREATE VERTEX command.

Viewing Unix Time

In addition to the formatted date and datetime, you can also view the underlying count from the Unix Epoch, using the asLong() method for records. For example,

orientdb> SELECT @RID, city, date.asLong() FROM #9:4

------+------+------------------------+
 @RID | city | date                   |
------+------+------------------------+
 #9:4 | Rome | -85889120400000        |
------+------+------------------------+

Meaning that, OrientDB represents the date of April 21, 753 BC, as -85889120400000 in Unix time. You can also work with dates directly as longs.

orientdb> CREATE VERTEX V SET city = "Rome", date = DATE(-85889120400000)
orientdb> SELECT @RID, city, date FROM V

-------+------+------------------------+
 @RID  | city | date                   |
-------+------+------------------------+
 #9:11 | Rome | 0753-04-21 00:00:00 BC |
-------+------+------------------------+

Use ISO 8601 Dates

According to ISO 8601, Combined date and time in UTC: 2014-12-20T00:00:00. To use this standard change the datetimeformat in the database:

ALTER DATABASE DATETIMEFORMAT "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"

Transactions

A transaction comprises a unit of work performed within a database management system (or similar system) against a database, and treated in a coherent and reliable way independent of other transactions. Transactions in a database environment have two main purposes:

• to provide reliable units of work that allow correct recovery from failures and keep a database consistent even in cases of system failure, when execution stops (completely or partially) and many operations upon a database remain uncompleted, with unclear status
• to provide isolation between programs accessing a database concurrently. If this isolation is not provided, the program's outcome are possibly erroneous.

A database transaction, by definition, must be atomic, consistent, isolated and durable. Database practitioners often refer to these properties of database transactions using the acronym ACID. --- Wikipedia

OrientDB is an ACID compliant DBMS.

NOTE: OrientDB keeps the transaction on client RAM, so the transaction size is affected by the available RAM (Heap memory) on JVM. For transactions involving many records, consider to split it in multiple transactions.

ACID properties

Atomicity

"Atomicity requires that each transaction is 'all or nothing': if one part of the transaction fails, the entire transaction fails, and the database state is left unchanged. An atomic system must guarantee atomicity in each and every situation, including power failures, errors, and crashes. To the outside world, a committed transaction appears (by its effects on the database) to be indivisible ("atomic"), and an aborted transaction does not happen." - WikiPedia

Consistency

"The consistency property ensures that any transaction will bring the database from one valid state to another. Any data written to the database must be valid according to all defined rules, including but not limited to constraints, cascades, triggers, and any combination thereof. This does not guarantee correctness of the transaction in all ways the application programmer might have wanted (that is the responsibility of application-level code) but merely that any programming errors do not violate any defined rules." - WikiPedia

OrientDB uses the MVCC to assure consistency. The difference between the management of MVCC on transactional and not-transactional cases is that with transactional, the exception rollbacks the entire transaction before to be caught by the application.

Look at this example:

Isolation

"The isolation property ensures that the concurrent execution of transactions results in a system state that would be obtained if transactions were executed serially, i.e. one after the other. Providing isolation is the main goal of concurrency control. Depending on concurrency control method, the effects of an incomplete transaction might not even be visible to another transaction." - WikiPedia

OrientDB has different levels of isolation based on settings and configuration:

• READ COMMITTED, the default and the only one available with remote protocol
• REPEATABLE READS, allowed only with plocal and memory protocols. This mode consumes more memory than READ COMMITTED, because any read, query, etc. keep the records in memory to assure the same copy on further access

To change default Isolation Level, use the Java API:

db.begin()
db.getTransaction().setIsolationLevel(OTransaction.ISOLATION_LEVEL.REPEATABLE_READ);

Using remote access all the commands are executed on the server, so out of transaction scope. Look below for more information.

Look at this examples:

|Sequence| Client/Thread 1 | Client/Thread 2 | |-----|-----|-------|------| |1| Begin of Transaction | | |2| read(x) | | |3| | Begin of Transaction | |4| | read(x) | |5| | write(x) | |6| | commit | |7| read(x) | | |8| commit | |

At operation 7 the client 1 continues to read the same version of x read in operation 2.

|Sequence| Client/Thread 1 | Client/Thread 2 | |-----|-----|-------|------| |1| Begin of Transaction | | |2| read(x) | | |3| | Begin of Transaction | |4| | read(y) | |5| | write(y) | |6| | commit | |7| read(y) | | |8| commit | |

At operation 7 the client 1 reads the version of y which was written at operation 6 by client 2. This is because it never reads y before.

Breaking of ACID properties when using remote protocol and Commands (SQL, Gremlin, JS, etc)

Transactions are client-side only until the commit. This means that if you're using the "remote" protocol the server can't see local changes.

In this scenario you can have different isolation levels with commands. This issue will be solved with OrientDB v3.0 where the transaction will be flushed to the server before to execute the command.

Durability

"Durability means that once a transaction has been committed, it will remain so, even in the event of power loss, crashes, or errors. In a relational database, for instance, once a group of SQL statements execute, the results need to be stored permanently (even if the database crashes immediately thereafter). To defend against power loss, transactions (or their effects) must be recorded in a non-volatile memory." - WikiPedia

Fail-over

An OrientDB instance can fail for several reasons:

• HW problems, such as loss of power or disk error
• SW problems, such as a Operating System crash
• Application problem, such as a bug that crashes your application that is connected to the Orient engine.

You can use the OrientDB engine directly in the same process of your application. This gives superior performance due to the lack of inter-process communication. In this case, should your application crash (for any reason), the OrientDB Engine also crashes.

If you're using an OrientDB Server connected remotely, if your application crashes the engine continue to work, but any pending transaction owned by the client will be rolled back.

Auto-recovery

At start-up the OrientDB Engine checks to if it is restarting from a crash. In this case, the auto-recovery phase starts which rolls back all pending transactions.

OrientDB has different levels of durability based on storage type, configuration and settings.

Transaction types

No Transaction

Default mode. Each operation is executed instantly.

Calls to begin(), commit() and rollback() have no effect.

Optimistic Transaction

This mode uses the well known Multi Version Control System (MVCC) by allowing multiple reads and writes on the same records. The integrity check is made on commit. If the record has been saved by another transaction in the interim, then an OConcurrentModificationException will be thrown. The application can choose either to repeat the transaction or abort it.

NOTE: OrientDB keeps the transaction on client RAM, so the transaction size is affected by the available RAM (Heap) memory on JVM. For transactions involving many records, consider to split it in multiple transactions.

With Graph API transaction begins automatically, with Document API is explicit by using the begin() method. With Graphs you can change the consistency level.

Example with Document API:

db.open("remote:localhost:7777/petshop");

try{
  db.begin(TXTYPE.OPTIMISTIC);
  ...
  // WRITE HERE YOUR TRANSACTION LOGIC
  ...
  db.commit();
}catch( Exception e ){
  db.rollback();
} finally{
  db.close();
}

In Optimistic transaction new records take temporary Record IDs to avoid to ask to the server a new Record ID every time. Temporary Record IDs have Cluster Id -1 and Cluster Position < -1. When a new transaction begun the counter is reset to -1:-2. So if you create 3 new records you'll have:

• -1:-2
• -1:-3
• -1:-4

At commit time, these temporary records Record IDs will be converted in the final ones.

Pessimistic Transaction

This mode is not yet supported by the engine.

Nested transactions and propagation

OrientDB doesn't support nested transaction. If further begin() are called after a transaction is already begun, then the current transaction keeps track of call stack to let to the final commit() call to effectively commit the transaction. Look at Transaction Propagation more information.

Record IDs

OrientDB uses temporary Record ID's with transaction as scope that will be transformed to finals once the transactions is successfully committed to the database. This avoid to ask for a free slot every time a client creates a record.

Tuning

In some situations transactions can improve performance, typically in the client/server scenario. If you use an Optimistic Transaction, the OrientDB engine optimizes the network transfer between the client and server, saving both CPU and bandwidth.

For further information look at Transaction tuning to know more.

Distributed environment

Transactions can be committed across a distributed architecture. Look at Distributed Transactions for more information.

Embed the Server

Embedding an OrientDB Server inside a Java application has several advantages and interesting features:

• Java application that runs embedded with the server can bypass the remote connection and use the database directly with local mode. local and remote connections against the same database can work in concurrency: OrientDB will synchronize the access.
• You can use the Console to control it
• You can use the OrientDB Studio
• You can replicate the database across distributed standalone or embedded servers

To embed an OrientDB Server inside a Java application you have to create the OServer object and use a valid configuration for it.

Requirements

In order to embed the server you need to include the following jar files in the classpath:

• orientdb-client-**.jar
• orientdb-core-**.jar
• orientdb-server-**.jar
• orientdb-tools-**.jar
• concurrentlinkedhashmap-lru-**.jar
• jna-**.jar
• lz4-java-**.jar

Starting from version 2.2, please set the MaxDirectMemorySize parameter. Setting this parameter is required. You can set it to a very high value, e.g. 512g (JVM setting):

-XX:MaxDirectMemorySize=512g

Setting MaxDirectMemorySize to a very high value should not concern you as it does not mean that OrientDB will consume all 512GB of memory. The size of direct memory consumed by OrientDB is limited by the size of the disk cache (variable storage.diskCache.bufferSize).

When you start the Server using the provided server.sh or server.bat scripts, MaxDirectMemorySize is set already by those scripts. But when you embed the Server it is required that you set MaxDirectMemorySize manually.

Note: if you are using a pom file, you may set MaxDirectMemorySize inside your pom in the following way:

<properties>
   <argLine>-XX:MaxDirectMemorySize=512g</argLine>
</properties>

If you start an embedded Server without setting this variable you will get a WARNING message similar to the following:

MaxDirectMemorySize JVM option is not set or has invalid value, that may cause out of memory errors

Include the commands you need

Even if most of the HTTP commands are auto registered assure to have all the commands you need. For example the static content must be registered. This is fundamental if you want to use OrientDB as Web Server providing static content like the Studio app:

<listener protocol="http" port-range="2480-2490" ip-address="0.0.0.0">
  <commands>
    <command implementation="com.orientechnologies.orient.server.network.protocol.http.command.get.OServerCommandGetStaticContent" pattern="GET|www GET|studio/ GET| GET|*.htm GET|*.html GET|*.xml GET|*.jpeg GET|*.jpg GET|*.png GET|*.gif GET|*.js GET|*.css GET|*.swf GET|*.ico GET|*.txt GET|*.otf GET|*.pjs GET|*.svg">
      <parameters>
        <entry value="Cache-Control: no-cache, no-store, max-age=0, must-revalidate\r\nPragma: no-cache" name="http.cache:*.htm *.html"/>
        <entry value="Cache-Control: max-age=120" name="http.cache:default"/>
      </parameters>
    </command>
  </commands>
</listener>

Use an embedded configuration

import com.orientechnologies.orient.server.OServerMain;

public class OrientDBEmbeddable {

 public static void main(String[] args) throws Exception {
  OServer server = OServerMain.create();
  server.startup(
   "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>"
   + "<orient-server>"
   + "<network>"
   + "<protocols>"
   + "<protocol name=\"binary\" implementation=\"com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary\"/>"
   + "<protocol name=\"http\" implementation=\"com.orientechnologies.orient.server.network.protocol.http.ONetworkProtocolHttpDb\"/>"
   + "</protocols>"
   + "<listeners>"
   + "<listener ip-address=\"0.0.0.0\" port-range=\"2424-2430\" protocol=\"binary\"/>"
   + "<listener ip-address=\"0.0.0.0\" port-range=\"2480-2490\" protocol=\"http\"/>"
   + "</listeners>"
   + "</network>"
   + "<users>"
   + "<user name=\"root\" password=\"ThisIsA_TEST\" resources=\"*\"/>"
   + "</users>"
   + "<properties>"
   + "<entry name=\"orientdb.www.path\" value=\"C:/work/dev/orientechnologies/orientdb/releases/1.0rc1-SNAPSHOT/www/\"/>"
   + "<entry name=\"orientdb.config.file\" value=\"C:/work/dev/orientechnologies/orientdb/releases/1.0rc1-SNAPSHOT/config/orientdb-server-config.xml\"/>"
   + "<entry name=\"server.cache.staticResources\" value=\"false\"/>"
   + "<entry name=\"log.console.level\" value=\"info\"/>"
   + "<entry name=\"log.file.level\" value=\"fine\"/>"
   //The following is required to eliminate an error or warning "Error on resolving property: ORIENTDB_HOME"
   + "<entry name=\"plugin.dynamic\" value=\"false\"/>"
   + "</properties>" + "</orient-server>");
  server.activate();
  }
}

Once the embedded server is running, clients can connect using the remote connection method. For example in the console, you can connect with:

connect remote:localhost:{port}/{db} {user} {password}
where:
  port     : the port that the binary server listens on
             (first free port from 2424-2430 according to the configuration above)
  db       : the database name to connect to (defaults to "db" and can be set using <entry name="server.database.path" value="db"/> in the configuration
  user     : the user to connect with (this is NOT the same as root user in the configuration)
  password : the user to connect with (this is NOT the same as root password in the configuration)

Use custom file for configuration

Use a regular File:

public class OrientDBEmbeddable {
  public static void main(String[] args) throws Exception {
      OServer server = OServerMain.create();
      server.startup(new File("/usr/local/temp/db.config"));
      server.activate();
  }
}

Use a stream for configuration

Use an InputStream from the class loader:

public class OrientDBEmbeddable {
  public static void main(String[] args) throws Exception {
      OServer server = OServerMain.create();
      server.startup(getClass().getResourceAsStream("db.config"));
      server.activate();
  }
}

Use a OServerConfiguration object for configuration

Or an InputStream from the class loader:

public class OrientDBEmbeddable {
  public static void main(String[] args) throws Exception {
      OServer server = OServerMain.create();
      OServerConfiguration cfg = new OServerConfiguration();
      // FILL THE OServerConfiguration OBJECT
      server.startup(cfg);
      server.activate();
  }
}

Shutdown

OrientDB Server creates some threads internally as non-daemon, so they run even if the main application exits. Use the OServer.shutdown() method to shutdown the server in soft way:

import com.orientechnologies.orient.server.OServerMain;

public class OrientDBEmbeddable {

  public static void main(String[] args) throws Exception {
    OServer server = OServerMain.create();
    server.startup(new File("/usr/local/temp/db.config"));
    server.activate();
    ...
    server.shutdown();
  }
}

Setting ORIENTDB_HOME

Some functionality will not work properly if the system property 'ORIENTDB_HOME' is not set. You can set it programmatically like this:

import com.orientechnologies.orient.server.OServerMain;

public class OrientDBEmbeddable {

  public static void main(String[] args) throws Exception {
    String orientdbHome = new File("").getAbsolutePath(); //Set OrientDB home to current directory
    System.setProperty("ORIENTDB_HOME", orientdbHome);

    OServer server = OServerMain.create();
    server.startup(cfg);
    server.activate();
  }
}

OrientDB Plugins

The OrientDB Server is a customizable platform to build powerful server component and applications.

Since the OrientDB server contains an integrated Web Server what about creating server side applications without the need to have a J2EE and Servlet container? By extending the server you can benefit of the best performance because you don't have many layers but the database and the application reside on the same JVM without the cost of the network and serialization of requests.

Furthermore you can package your application together with the OrientDB server to distribute just a ZIP file containing the entire Application, Web server and Database.

To customize the OrientDB server you have two powerful tools:

• Handlers
• Custom commands

To debug the server while you develop new feature follow Debug the server.

Handlers (Server Plugins)

Handlers are plug-ins and starts when OrientDB starts.

To create a new handler create the class and register it in the OrientDB server configuration.

Create the Handler class

A Handler must implements the OServerPlugin interface or extends the OServerPluginAbstract abstract class.

Below an example of a handler that print every 5 seconds a message if the "log" parameters has been configured to be "true":

package orientdb.test;

public class PrinterHandler extends OServerPluginAbstract {
  private boolean	log = false;

  public void config(OServer oServer, OServerParameterConfiguration[] iParams) {
    for (OServerParameterConfiguration p : iParams) {
      if (p.name.equalsIgnoreCase("log"))
        log = true;
    }

    Orient.getTimer().schedule( new TimerTask() {
      public void run() {
        if( log )
          System.out.println("It's the PrinterHandler!");
      }
    }, 5000, 5000);
  }

  public String getName() {
    return "PrinterHandler";
  }
}

Register the handler

Once created, register it to the server configuration in orientdb-server-config.xml file:

<orient-server>
  <handlers>
    <handler class="orientdb.test.PrinterHandler">
      <parameters>
        <parameter name="log" value="true"/>
      </parameters>
    </handler>
  </handlers>
  ...

Note that you can specify arbitrary parameters in form of name and value. Those parameters can be read by the config() method. In this example a parameter "log" is read. Look upon to the example of handler to know how to read parameters specified in configuration.

Steps to register a function as a Plugin in OrientDB

In this case we'll create a plugin that only registers one function in OrientDB: pow (returns the value of the first argument raised to the power of the second argument). We'll also support Modular exponentiation.

The syntax will be pow(<base>, <power> [, <mod>]).

• you should have a directory structure like this

    .
    ├─ src
    |   └─ main
    |       ├─ assembly
    |       |   └─ assembly.xml 
    |       ├─ java
    |       |   └─ com
    |       |       └─ app
    |       |           └─ OPowPlugin.java
    |       └─ resources
    |           └─ plugin.json 
    |
    └─ pom.xml 

OPowPlugin.java

package com.app;

import com.orientechnologies.common.log.OLogManager;
import com.orientechnologies.orient.core.command.OCommandContext;
import com.orientechnologies.orient.core.db.record.OIdentifiable;
import com.orientechnologies.orient.core.sql.OSQLEngine;
import com.orientechnologies.orient.core.sql.functions.OSQLFunctionAbstract;
import com.orientechnologies.orient.server.OServer;
import com.orientechnologies.orient.server.config.OServerParameterConfiguration;
import com.orientechnologies.orient.server.plugin.OServerPluginAbstract;
import java.util.ArrayList;
import java.util.List;

public class OPowPlugin extends OServerPluginAbstract {

    public OPowPlugin() {
    }

    public String getName() {
        return "pow-plugin";
    }

    public void startup() {
        super.startup();
        OSQLEngine.getInstance().registerFunction("pow", new OSQLFunctionAbstract("pow", 2, 3) {

            public String getSyntax() {
                return "pow(<base>, <power> [, <mod>])";
            }

            public Object execute(Object iThis, OIdentifiable iCurrentRecord, Object iCurrentResult, final Object[] iParams, OCommandContext iContext) {
                if (iParams[0] == null || iParams[1] == null) {
                    return null;
                }
                if (!(iParams[0] instanceof Number) || !(iParams[1] instanceof Number)) {
                    return null;
                }

                final long base = ((Number) iParams[0]).longValue();
                final long power = ((Number) iParams[1]).longValue();

                if (iParams.length == 3) { // modular exponentiation
                    if (iParams[2] == null) {
                        return null;
                    }
                    if (!(iParams[2] instanceof Number)) {
                        return null;
                    }

                    final long mod = ((Number) iParams[2]).longValue();
                    if (power < 0) {
                        OLogManager.instance().warn(this, "negative numbers as exponent are not supported");
                    }
                    return modPow(base, power, mod);
                }

                return power > 0 ? pow(base, power) : 1D / pow(base, -power);
            }
        });
        OLogManager.instance().info(this, "pow function registered");
    }

    private double pow(long base, long power) {
        double r = 1;
        List<Boolean> bits = bits(power);
        for (int i = bits.size() - 1; i >= 0; i--) {
            r *= r;
            if (bits.get(i)) {
                r *= base;
            }
        }

        return r;
    }

    private double modPow(long base, long power, long mod) {
        double r = 1;
        List<Boolean> bits = bits(power);
        for (int i = bits.size() - 1; i >= 0; i--) {
            r = (r * r) % mod;
            if (bits.get(i)) {
                r = (r * base) % mod;
            }
        }

        return r;
    }

    private List<Boolean> bits(long n) {
        List<Boolean> bits = new ArrayList();
        while (n > 0) {
            bits.add(n % 2 == 1);
            n /= 2;
        }

        return bits;
    }

    public void config(OServer oServer, OServerParameterConfiguration[] iParams) {

    }

    public void shutdown() {
        super.shutdown();
    }
}

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.app</groupId>
    <artifactId>pow-plugin</artifactId>
    <version>2.0.7</version>
    <packaging>jar</packaging>

    <name>pow-plugin</name>

    <properties>
        <orientdb.version>2.0.7</orientdb.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.4</version>
                <configuration>
                    <descriptors>
                        <descriptor>src/main/assembly/assembly.xml</descriptor>
                    </descriptors>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <!-- this is used for inheritance merges -->
                        <phase>package</phase>
                        <!-- bind to the packaging phase -->
                        <goals>
                            <goal>single</goal>
                        </goals>
                        <configuration></configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                </configuration>
            </plugin>
        </plugins>
    </build>
    
    <dependencies>
        <dependency>
            <groupId>com.orientechnologies</groupId>
            <artifactId>orientdb-core</artifactId>
            <version>${orientdb.version}</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>com.orientechnologies</groupId>
            <artifactId>orientdb-server</artifactId>
            <version>${orientdb.version}</version>
            <scope>compile</scope>
        </dependency>
    </dependencies>
</project>

assembly.xml

<assembly>
    <id>dist</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
            <outputDirectory/>
            <unpack>true</unpack>
            <includes>
                <include>${groupId}:${artifactId}</include>
            </includes>
        </dependencySet>
    </dependencySets>
</assembly>

plugin.json

{
	"name" : "pow-plugin",
	"version" : "2.0.7",
	"javaClass": "com.app.OPowPlugin",
	"parameters" : {},
	"description" : "The Pow Plugin",
	"copyrights" : "No copyrights"
}

• Build the project and then:

cp target/pow-plugin-2.0.7-dist.jar $ORIENTDB_HOME/plugins/

You should see the following in OrientDB server log:

INFO  Installing dynamic plugin 'pow-plugin-2.0.7-dist.jar'... [OServerPluginManager]
INFO  pow function registered [OPowPlugin]

And now you can:

orientdb {db=Pow}> select pow(2,10)     

----+------+------
#   |@CLASS|pow   
----+------+------
0   |null  |1024.0
----+------+------

orientdb {db=Pow}> select pow(2,10,5)

----+------+----
#   |@CLASS|pow 
----+------+----
0   |null  |4.0 
----+------+----

This small project is available here.

Creating a distributed change manager

As more complete example let's create a distributed record manager by installing hooks to all the server's databases and push these changes to the remote client caches.

public class DistributedRecordHook extends OServerHandlerAbstract implements ORecordHook {
  private boolean log = false;

  public void config(OServer oServer, OServerParameterConfiguration[] iParams) {
    for (OServerParameterConfiguration p : iParams) {
      if (p.name.equalsIgnoreCase("log"))
        log = true;
    }
  }

  public void onAfterClientRequest(final OClientConnection iConnection, final byte iRequestType) {
    if (iRequestType == OChannelBinaryProtocol.REQUEST_DB_OPEN)
      iConnection.database.registerHook(this);
    else if (iRequestType == OChannelBinaryProtocol.REQUEST_DB_CLOSE)
      iConnection.database.unregisterHook(this);
  }

  public boolean onTrigger(TYPE iType, ORecord<?> iRecord) {
    try {
      if (log)
        System.out.println("Broadcasting record: " + iRecord + "...");

      OClientConnectionManager.instance().broadcastRecord2Clients((ORecordInternal<?>) iRecord, null);
    } catch (Exception e) {
      e.printStackTrace();
    }
    return false;
  }

  public String getName() {
    return "DistributedRecordHook";
  }
}

Custom commands

Custom commands are useful when you want to add behavior or business logic at the server side.

A Server command is a class that implements the OServerCommand interface or extends one of the following abstract classes:

• OServerCommandAuthenticatedDbAbstract if the command requires an authentication at the database
• OServerCommandAuthenticatedServerAbstract if the command requires an authentication at the server

The Hello World Web

To learn how to create a custom command, let's begin with a command that just returns "Hello world!".

OrientDB follows the convention that the command name is:

OServerCommand<method><name> Where:

• method is the HTTP method and can be: GET, POST, PUT, DELETE
• name is the command name

In our case the class name will be "OServerCommandGetHello". We want that the use must be authenticated against the database to execute it as any user.

Furthermore we'd like to receive via configuration if we must display the text in Italic or not, so for this purpose we'll declare a parameter named "italic" of type boolean (true or false).

package org.example;

public class OServerCommandGetHello extends OServerCommandAuthenticatedDbAbstract {
  // DECLARE THE PARAMETERS
  private boolean italic = false;

  public OServerCommandGetHello(final OServerCommandConfiguration iConfiguration) {
    // PARSE PARAMETERS ON STARTUP
    for (OServerEntryConfiguration par : iConfiguration.parameters) {
      if (par.name.equals("italic")) {
        italic = Boolean.parseBoolean(par.value);
      }
    }
  }

  public boolean execute(final OHttpRequest iRequest, OHttpResponse iResponse) throws Exception {
    // CHECK THE SYNTAX. 3 IS THE NUMBER OF MANDATORY PARAMETERS
    String[] urlParts = checkSyntax(iRequest.url, 3, "Syntax error: hello/<database>/<name>");

    // TELLS TO THE SERVER WHAT I'M DOING (IT'S FOR THE PROFILER)
    iRequest.data.commandInfo = "Salutation";
    iRequest.data.commandDetail = "This is just a test";

    // GET THE PARAMETERS
    String name = urlParts[2];

    // CREATE THE RESULT
    String result = "Hello " + name;
    if (italic) {
      result = "<i>" + result + "</i>";
    }

    // SEND BACK THE RESPONSE AS TEXT
    iResponse.send(OHttpUtils.STATUS_OK_CODE, "OK", null, result, OHttpUtils.CONTENT_TEXT_PLAIN);

    // RETURN ALWAYS FALSE, UNLESS YOU WANT TO EXECUTE COMMANDS IN CHAIN
    return false;
  }

  public String[] getNames() {
    return new String[]{"GET|hello/*","POST|hello/*"};
  }
}

Once created the command you need to register them through the orientdb-server-config.xml file. Put a new tag <command> under the tag commands of <listener> with attribute protocol="http":

  ...
  <listener protocol="http" port-range="2480-2490" ip-address="0.0.0.0">
    <commands>
      <command implementation="org.example.OServerCommandGetHello" pattern="GET|hello/*">
        <parameters>
          <entry name="italic" value="true"/>
        </parameters>
      </command>
    </commands>
  </listener>

Where:

• implementation is the full class name of the command
• pattern is how the command is called in the format: <HTTP-method>|<name>. In this case it's executed on HTTP GET with the URL: /<name>
• parameters specify parameters to pass to the command on startup
• entry is the parameter pair name/value

To test it open a browser at this address:

http://localhost:2480/hello/<your-db-name>/Luca

You will see:

Hello Luca

Complete example

Below a more complex example taken by official distribution. It is the command that executes queries via HTTP. Note how to get a database instance to execute operation against the database:

public class OServerCommandGetQuery extends OServerCommandAuthenticatedDbAbstract {
  private static final String[] NAMES = { "GET|query/*" };

  public boolean execute(OHttpRequest iRequest, OHttpResponse iResponse) throws Exception {
    String[] urlParts = checkSyntax(
        iRequest.url,
        4,
        "Syntax error: query/<database>/sql/<query-text>[/<limit>][/<fetchPlan>].<br/>Limit is optional and is setted to 20 by default. Set expressly to 0 to have no limits.");

    int limit = urlParts.length > 4 ? Integer.parseInt(urlParts[4]) : 20;
    String fetchPlan = urlParts.length > 5 ? urlParts[5] : null;
    String text = urlParts[3];

    iRequest.data.commandInfo = "Query";
    iRequest.data.commandDetail = text;

    ODatabaseDocumentTx db = null;

    List<OIdentifiable> response;

    try {
      db = getProfiledDatabaseInstance(iRequest);
      response = (List<OIdentifiable>) db.command(new OSQLSynchQuery<OIdentifiable>(text, limit).setFetchPlan(fetchPlan)).execute();

    } finally {
      if (db != null) {
        db.close();
      }
    }

    iResponse.writeRecords(response, fetchPlan);
    return false;
  }

  public String[] getNames() {
    return NAMES;
  }
}

Include JARS in the classpath

If your extensions need additional libraries put the additional jar files under the /lib folder of the server installation.

Debug the server

To debug your plugin you can start your server in debug mode.

Automatic Backup Server Plugin

Using this server plugin, OrientDB executes regular backups on the databases. It implements the Java class:

com.orientechnologies.orient.server.handler.OAutomaticBackup

Plugin Configuration

Beginning with version 2.2, OrientDB manages the server plugin configuration from a separate JSON. You can update this file manually or through OrientDB Studio.

To enable automatic backups, use the following <handler> section in the config/orientdb-server-config.xml configuration file:

<!-- AUTOMATIC BACKUP, TO TURN ON SET THE 'ENABLED' PARAMETER TO 'true' -->
<handler class="com.orientechnologies.orient.server.handler.OAutomaticBackup">
    <parameters>
        <parameter name="enabled" value="false"/>
        <!-- LOCATION OF JSON CONFIGURATION FILE -->
        <parameter name="config" value="${ORIENTDB_HOME}/config/automatic-backup.json"/>
    </parameters>
</handler>

This section tells the OrientDB server to read the file at $ORIENTDB_HOME/config/automatic-backup.json for the automatic backup configuration.

{
  "enabled": true,
  "mode": "FULL_BACKUP",
  "exportOptions": "",
  "delay": "4h",
  "firstTime": "23:00:00",
  "targetDirectory": "backup",
  "targetFileName": "${DBNAME}-${DATE:yyyyMMddHHmmss}.zip",
  "compressionLevel": 9,
  "bufferSize": 1048576
}

• "enabled" Defines whether it uses automatic backups. The supported values are:
  • true Enables automatic backups.
  • false Disables automatic backups. This is the default setting.
• "mode" Defines the backup mode. The supported values are:
  • "FULL_BACKUP" Executes a full backup. Prior to version 2.2, this was the only mode available. This operation blocks the database.
  • "INCREMENTAL_BACKUP" Executes an incremental backup. This is available only in the Enterprise Edition. It uses one directory per database. This operation doesn't block the database.
  • "EXPORT" Executes an database export, using gziped JSON format. This operation is not blocking.
• "exportOptions" Defines export options to use with that mode. This feature was introduced in version 2.2.
• "delay" Defines the delay time for each backup. Supports the following suffixes:
  • ms Delay measured in milliseconds.
  • s Delay measured in seconds.
  • m Delay measured in minutes.
  • h Delay measured in hours.
  • d Delay measured in days.
• "firstTime" Defines when to initiate the first backup in the schedule. It uses the format of HH:mm:ss in the GMT time zone, on the current day.
• "targetDirectory" Defines the target directory to write backups. By default, it is set to the backup/ directory.
• "targetFileName" Defines the target filename. This parameter supports the use of the following variables, (that is, "${DBNAME}-backup.zip" produces mydatabase-backup.zip):
  • ${DBNAME} Renders the database name.
  • ${DATE} Renders the current date, using the Java DateTime syntax format.
• "dbInclude" Defines in a list the databases to include in the automatic backups. If empty, it backs up all databases.
• "dbExclude" Defines in a list the databases to exclude from the automatic backups.
• "bufferSize" Defines the in-memory buffer sizes to use in compression. By default, it is set to 1MB. Larger buffers mean faster backups, but they in turn consume more RAM.
• "compressionLevel" Defines the compression level for the resulting ZIP file. By default it is set to the maximum level of 9. Set it to a lower value if you find that the backup takes too much time.

Legacy Plugin Configuration

In versions prior to 2.2, the only option in configuring automatic backups is to use the config/orientdb-server-config.xml configuration file. Beginning with version 2.2 you can manage automatic backup configuration through a separate JSON file or use the legacy approach.

The example below configures automatic backups/exports on the database as a Server Plugin.

<!-- AUTOMATIC BACKUP, TO TURN ON SET THE 'ENABLED' PARAMETER TO 'true' -->
<handler class="com.orientechnologies.orient.server.handler.OAutomaticBackup">
  <parameters>
    <parameter name="enabled" value="false" />
     <!-- CAN BE: FULL_BACKUP, INCREMENTAL_BACKUP, EXPORT -->
     <parameter name="mode" value="FULL_BACKUP"/>
     <!-- OPTION FOR EXPORT -->
     <parameter name="exportOptions" value=""/>
    <parameter name="delay" value="4h" />
    <parameter name="target.directory" value="backup" />
    <!-- ${DBNAME} AND ${DATE:} VARIABLES ARE SUPPORTED -->
    <parameter name="target.fileName" value="${DBNAME}-${DATE:yyyyMMddHHmmss}.zip" />
    <!-- DEFAULT: NO ONE, THAT MEANS ALL DATABASES. 
	     USE COMMA TO SEPARATE MULTIPLE DATABASE NAMES -->
    <parameter name="db.include" value="" />
    <!-- DEFAULT: NO ONE, THAT MEANS ALL DATABASES. 
	     USE COMMA TO SEPARATE MULTIPLE DATABASE NAMES -->
    <parameter name="db.exclude" value="" />
    <parameter name="compressionLevel" value="9"/>
    <parameter name="bufferSize" value="1048576"/>
  </parameters>
</handler>

• enabled Defines whether it uses automatic backups. Supported values are:
  • true Enables automatic backups.
  • false Disables automatic backups. This is the default setting.
• mode/> Defines the backup mode. Supported values are:
  • FULL_BACKUP Executes a full backup. For versions prior to 2.2, this is the only option available. This operation blocks the database.
  • INCREMENTAL_BACKUP Executes an incremental backup. Uses one directory per database. This operation doesn't block the database.
  • *EXPORT Executes an export of the database in gzipped JSON format, instead of a backup. This operation doesn't block the database.
• exportOptions Defines export options to use with that mode. This feature was introduced in version 2.2.
• delay Defines the delay time. Supports the following suffixes:
  • ms Delay measured in milliseconds.
  • s Delay measured in seconds.
  • m Delay measured in minutes.
  • h Delay measured in hours.
  • d Delay measured in days.
• firstTime Defines when to initiate the first backup in the schedule. It uses the format of HH:mm:ss in the GMT time zone, on the current day.
• target.directory Defines the target directory to write backups. By default, it is set to the backup/ directory.
• target.fileName Defines the target file name. The parameter supports the use of the following variables, (that is, <parameter name="target.filename" value="${DBNAME}-backup.zip"/> produces a mydatabase-backup.zip file).
  • ${DBNAME} Renders the database name.
  • ${DATE} Renders the current date, using the Java DateTime syntax format.
• db.include Defines in a list the databases to include in the automatic backups. If left empty, it backs up all databases.
• db.exclude Defines in a list the databases to exclude from automatic backups.
• bufferSize Defines the in-memory buffer sizes to use in compression. By default it is set to 1MB. Larger buffers mean faster backups, but use more RAM. This feature was introduced in version 1.7.
• compressionLevel Defines the compression level for the resulting ZIP file. By default, it is set to the maximum level of 9. Set it to a lower value if you find that the backup takes too much time.

SysLog Plugin

Java class implementation:

com.orientechnologies.security.syslog.ODefaultSyslog

Available since: v. 2.2.0.

Introduction

Allows sending event logs to the Operating System's SYSLOG daemon.

Configuration

This plugin is configured as a Server plugin. The plugin can be easily configured by changing parameters in the orientdb-server-config.xml file.:

Default configuration in orientdb-server-config.xml. Example:

<!-- SYS LOG CONNECTOR, TO TURN ON SET THE 'ENABLED' PARAMETER TO 'true' -->
<handler class="com.orientechnologies.security.syslog.ODefaultSyslog">
    <parameters>
        <parameter name="enabled" value="true"/>
        <parameter name="debug" value="false"/>
        <parameter name="hostname" value="localhost"/>
        <parameter name="port" value="514"/>
        <parameter name="appName" value="OrientDB"/>
    </parameters>
</handler>

Usage

Look at Security Config.

Mail Plugin

Java class implementation:

com.orientechnologies.orient.server.plugin.mail.OMailPlugin

Available since: v. 1.2.0.

Introduction

Allows to send (and in future read) emails.

Configuration

This plugin is configured as a Server handler. The plugin can be configured in easy way by changing parameters:

Default configuration in orientdb-server-config.xml. Example:

<!-- MAIL, TO TURN ON SET THE 'ENABLED' PARAMETER TO 'true' -->
<handler
class="com.orientechnologies.orient.server.plugin.mail.OMailPlugin">
  <parameters>
    <parameter name="enabled" value="true" />
    <!-- CREATE MULTIPLE PROFILES WITH profile.<name>... -->
    <parameter name="profile.default.mail.smtp.host" value="smtp.gmail.com"/>
    <parameter name="profile.default.mail.smtp.port" value="587" />
    <parameter name="profile.default.mail.smtp.auth" value="true" />
    <parameter name="profile.default.mail.smtp.starttls.enable" value="true" />
    <parameter name="profile.default.mail.from" value="test@gmail.com" />
    <parameter name="profile.default.mail.smtp.user" value="test@gmail.com" />
    <parameter name="profile.default.mail.smtp.password" value="mypassword" />
    <parameter name="profile.default.mail.date.format" value="yyyy-MM-dd HH:mm:ss" />
  </parameters>
</handler>

Usage

The message is managed as a map of properties containing all the fields those are part of the message.

Supported message properties:

From Server-Side Functions

The Email plugin install a new variable in the server-side function's context: "mail". "profile" attribute is the profile name in configuration.

Example to send an email writing a function in JS:

mail.send({
      profile : "default",
      to: "orientdb@ruletheworld.com",
      cc: "yoda@starwars.com",
      bcc: "darthvader@starwars.com",
      subject: "The EMail plugin works",
      message : "Sending email from OrientDB Server is so powerful to build real web applications!"
    });

On Nashorn (>= Java8) the mapping of JSON to Map is not implicit. Use this:

mail.send( new java.util.HashMap{
      profile : "default",
      to: "orientdb@ruletheworld.com",
      cc: "yoda@starwars.com",
      bcc: "darthvader@starwars.com",
      subject: "The EMail plugin works",
      message : "Sending email from OrientDB Server is so powerful to build real web applications!"
  });

From Java

OMailPlugin plugin = OServerMain.server().getPlugin("mail");

Map<String, Object> message = new HashMap<String, Object>();
message.put("profile", "default");
message.put("to",      "orientdb@ruletheworld.com");
message.put("cc",      "yoda@starts.com,yoda-beach@starts.com");
message.put("bcc",     "darthvader@starwars.com");
message.put("subject", "The EMail plugin works");
message.put("message", "Sending email from OrientDB Server is so powerful to build real web applications!");

plugin.send(message);

JMX plugin

Java class implementation:

com.orientechnologies.orient.server.handler.OJMXPlugin

Available since: v. 1.2.0.

Introduction

Expose the OrientDB server configuration through JMX protocol. This task is configured as a Server handler. The task can be configured in easy way by changing parameters:

• enabled: true to turn on, false (default) is turned off
• profilerManaged: manage the Profiler instance

Default configuration in orientdb-server-config.xml

<!-- JMX SERVER, TO TURN ON SET THE 'ENABLED' PARAMETER TO 'true' -->
<handler class="com.orientechnologies.orient.server.handler.OJMXPlugin">
  <parameters>
    <parameter name="enabled" value="false" />
    <parameter name="profilerManaged" value="true" />
  </parameters>
</handler>

Rexster

Rexster provides a RESTful shell to any Blueprints-complaint graph database. This HTTP web service provides: a set of standard low-level GET, POST, and DELETE methods, a flexible extension model which allows plug-in like development for external services (such as ad-hoc graph queries through Gremlin), and a browser-based interface called The Dog House.

A graph database hosted in the OrientDB can be configured in Rexster and then accessed using the standard RESTful interface powered by the Rexster web server.

Installation

You can get the latest stable release of Rexster from its Download Page. The latest stable release when this page was last updated was 2.5.0.

Or you can build a snapshot by executing the following Git and Maven commands:

git clone https://github.com/tinkerpop/rexster.git
cd rexster
mvn clean install

Rexster is distributed as a zip file (also the building process creates a zip file) hence the installation consist of unzipping the archive in a directory of your choice. In the following sections, this directory is referred to as $REXSTER_HOME.

After unzipping the archive, you should copy orient-client.jar and orient-enterprise.jar in $REXSTER_HOME/ext. Make sure you use the same version of OrientDB as those used by Rexster. For example Rexster 2.5.0 uses OrientDB 1.7.6.

You can find more details about Rexster installation at the Getting Started page.

Configuration

Refer to Rexster's Configuration page and OrientDB specific configuration page for the latest details.

Synopsis

The Rexster configuration file rexster.xml is used to configure parameters such as: TCP ports used by Rexster server modules to listen for incoming connections; character set supported by the Rexster REST requests and responses; connection parameters of graph instances.

In order to configure Rexster to connect to your OrientDB graph, locate the rexster.xml in the Rexster directory and add the following snippet of code:

<rexster>
  ...
  <graphs>
    ...
    <graph>
      <graph-enabled>true</graph-enabled>
      <graph-name>my-orient-graph</graph-name>
      <graph-type>orientgraph</graph-type>
      <graph-file>url-to-your-db</graph-file>
      <properties>
        <username>user</username>
        <password>pwd</password>
      </properties>
    </graph>
  ...
  </graphs>
</rexster>

In the configuration file, there could be a sample graph element for an OrientDB instance (<graph-name>orientdbsample<graph-name>): you might edit it according to your needs.

The <graph-name> element must be unique within the list of configured graphs and reports the name used to identify your graph. The <graph-enabled> element states whether the graph should be loaded and managed by Rexster. Setting its contents to false will prevent that graph from loading to Rexster; setting explicitly to true the graph will be loaded. The <graph-type> element reports the type of graph by using an identifier (orientgraph for an OrientDB Graph instance) or the full name of the class that implements the GraphConfiguration interface (com.tinkerpop.rexster.OrientGraphConfiguration for an OrientDB Graph).

The <graph-file> element reports the URL to the OrientDB database Rexster is expected to connect to:

• plocal:*path-to-db*, if the graph can be accessed over the file system (e.g. plocal:/tmp/graph/db)
• remote:*url-to-db*, if the graph can be accessed over the network and/or if you want to enable multiple accesses to the graph (e.g. remote:localhost/mydb)
• memory:*db-name*, if the graph resides in memory only. Updates to this kind of graph are never persistent and when the OrientDB server ends the graph is lost

The <username> and <password> elements reports the credentials to access the graph (e.g. admin admin).

Run

Note: only Rexster 0.5-SNAPSHOT and further releases work with OrientDB GraphEd
In this section we present a step-by-step guide to Rexster-ify an OrientDB graph.
We assume that:

• you created a Blueprints enabled graph called orientGraph using the class com.tinkerpop.blueprints.pgm.impls.orientdb.OrientGraph
• you inserted in the Rexster configuration file a <graph> element with the <graph-name> element set to my-orient-graph and the graph-file element set to remote:orienthost/orientGraph (if you do not remember how to do this, go back to the Configuration section).

1. Be sure that the OrientDB server is running and you have properly configured the <graph-file> location and the access credentials of your graph.
2. Execute the startup script ($REXSTER_HOME/bin/rexster.bat or $REXSTER_HOME/bin/rexster.sh)
3. The shell console appears and you should see the following log message (line 10 states that the OrientDB graph instance has been loaded):

[INFO] WebServer - .:Welcome to Rexster:.
[INFO] GraphConfigurationContainer - Graph emptygraph - tinkergraph[vertices:0 edges:0] loaded
[INFO] RexsterApplicationGraph - Graph [tinkergraph] - configured with allowable namespace [tp:gremlin]
[INFO] GraphConfigurationContainer - Graph tinkergraph - tinkergraph[vertices:6 edges:6] loaded
[INFO] RexsterApplicationGraph - Graph [tinkergraph-readonly] - configured with allowable namespace [tp:gremlin]
[INFO] GraphConfigurationContainer - Graph tinkergraph-readonly - (readonly)tinkergraph[vertices:6 edges:6] loaded
[INFO] RexsterApplicationGraph - Graph [gratefulgraph] - configured with allowable namespace [tp:gremlin]
[INFO] GraphConfigurationContainer - Graph gratefulgraph - tinkergraph[vertices:809 edges:8049] loaded
[INFO] GraphConfigurationContainer - Graph sailgraph - sailgraph[memorystore] loaded
[INFO] GraphConfigurationContainer - Graph my-orient-graph - orientgraph[remote:orienthost/orientGraph] loaded
[INFO] GraphConfigurationContainer - Graph neo4jsample -  not enabled and not loaded.
[INFO] GraphConfigurationContainer - Graph dexsample -  not enabled and not loaded.
[INFO] MapResultObjectCache - Cache constructed with a maximum size of 1000
[INFO] WebServer - Web Server configured with com..sun..jersey..config..property..packages: com.tinkerpop.rexster
[INFO] WebServer - No servlet initialization parameters passed for configuration: admin-server-configuration
[INFO] WebServer - Rexster Server running on: [http://localhost:8182]
[INFO] WebServer - Dog House Server running on: [http://localhost:8183]
[INFO] ShutdownManager$ShutdownSocketListener - Bound shutdown socket to /127.0.0.1:8184. Starting listener thread for shutdown requests.

• Now you can use Rexster REST API and The Dog House web application to retrieve and modify the data stored in the OrientDB graph.

Gephi Visual Tool

Introduction

Gephi is a visual tool to manipulate and analyze graphs. http://gephi.org is an Open Source project. Take a look at the amazing features.

Gephi can be used to analyze graphs extracted from OrientDB. There are 2 level of integration:

• the Streaming plugin that calls OrientDB server via HTTP. OrientDB exposes the new "/gephi" command in HTTP GET method that executes a query and returns the result set in "gephi" format.
• Gephi importer for Blueprints

In this mini guide we will take a look at the first one: the streaming plugin.

For more information:

• Gephi Graph Streaming format
• Tutorial video

Getting started

Before to start assure you've OrientDB 1.1.0-SNAPSHOT or greater.

Download and install

1. To download Gephi goto: http://gephi.org/users/download/
2. Install it, depends on your OS
3. Run Gephi
4. Click on the menu Tools -> Plugins
5. Click on the tab Available Plugins
6. Select the plugin Graph Streaming, click on the Install button and wait the plugin is installed

Import a graph in Gephi

Before to import a graph assure a OrientDB server instance is running somewhere. For more information watch this video.

1. Go to the Overview view (click on Overview top left button)
2. Click on the Streaming tab on the left
3. Click on the big + green button
4. Insert as Source URL the query you want to execute. Example: http://localhost:2480/gephi/demo/sql/select%20from%20v/100 (below more information about the syntax of query)
5. Select as Stream type the JSON format (OrientDB talks in JSON)
6. Enable the Use Basic Authentication and insert the user and password of OrientDB database you want to access. The default user is "admin" as user and password
7. Click on OK button

Executing a query

The OrientDB's "/gephi" HTTP command allow to execute any query. The format is:

http://<host>:<port>/gephi/<database>/<language>/<query>[/<limit>]

Where:

• host is the host name or the ip address where the OrientDB server is running. If you're executing OrientDB on the same machine where Gephi is running use "localhost"
• port is the port number where the OrientDB server is running. By default is 2480.
• database is the database name
• language
• query, the query text following the URL encoding rules. For example to use the spaces use %20, so the query select from v becomes select%20from%20v
• limit, optional, set the limit of the result set. If not defined 20 is taken by default. -1 means no limits

SQL Graph language

To use the OrientDB's SQL language use sql as language. For more information look at the SQL-Syntax.

For example, to return the first 1,000 vertices (class V) with outgoing connections the query would be:

SELECT FROM V WHERE out.size() > 0

Executed on "localhost" against the "demo" database + encoding becomes:

http://localhost:2480/gephi/demo/sql/select%20from%20V%20where%20out.size()%20%3E%200/1000

GREMLIN language

To use the powerful GREMLIN language to retrieve the graph or a portion of it use gremlin as language. For more information look at the GREMLIN syntax.

For example, to return the first 100 vertices:

g.V[0..99]

Executed on "localhost" against the "demo" database + encoding becomes:

http://localhost:2480/gephi/demo/gremlin/g.V%5B0..99%5D/-1

spider-box

spider-box is not really a "plug-in", but more a quick way to set up an environment to play with OrientDB in a local VM. It requires a virtualization system like Virtualbox, VMWare Fusion or Parallels and the provisioning software Vagrant.

Once installed, you can very quickly start playing with the newest version of OrientDB Studio or the console. Or even start developing software with OrientDB as the database.

spider-box is configured mainly to build a PHP development environment. But, since it is built on Puphpet, you can easily change the configuration, so Python or even node.js is also installed. Ruby is installed automatically.

If you have questions about changing configuration or using spider-box, please do ask in an issue in the spider-box repo.

Have fun playing with OrientDB and spider-box!

Note: Neo4j and Gremlin Server are also installed, when you vagrant up spider-box.

Plugins

If you're looking for drivers or JDBC connector go to Programming-Language-Bindings.

• OrientDB Spring Data is the official Spring Data Plugin for both Graph and Document APIs
• spring-orientdb is an attempt to provide a PlatformTransactionManager for OrientDB usable with the Spring Framework, in particular with @Transactional annotation. Apache 2 license
  
• Spring Session OrientDB is a Spring Session extension for OrientDB.

• Play Framework 2.1 PLAY-WITH-ORIENTDB plugin
• Play Framework 2.1 ORIGAMI plugin
• Play Framework 1.x ORIENTDB plugin
• Frames-OrientDB Plugin Play Framework 2.x Frames-OrientDB plugin is a Java O/G mapper for the OrientDB with the Play! framework 2. It is used with the TinkerPop Frames for O/G mapping.

With proper mark-up/logic separation, a POJO data model, and a refreshing lack of XML, Apache Wicket makes developing web-apps simple and enjoyable again. Swap the boilerplate, complex debugging and brittle code for powerful, reusable components written with plain Java and HTML.

Guice (pronounced 'juice') is a lightweight dependency injection framework for Java 6 and above, brought to you by Google. OrientDB Guice plugin allows to integrate OrientDB inside Guice. Features:

• Integration through guice-persist (UnitOfWork, PersistService, @Transactional, dynamic finders supported)
• Support for document, object and graph databases
• Database types support according to classpath (object and graph db support activated by adding jars to classpath)
• Auto mapping entities in package to db scheme or using classpath scanning to map annotated entities
• Auto db creation
• Hooks for schema migration and data initialization extensions
• All three database types may be used in single unit of work (but each type will use its own transaction)

Vert.x is a lightweight, high performance application platform for the JVM that's designed for modern mobile, web, and enterprise applications. Vert.x Persistor Module for Tinkerpop-compatible Graph Databases like OrientDB.

Gephi Visual tool usage with OrientDB and the Blueprints importer

OrientDB session store for Connect

Chef

Apache Tomcat realm plugin by Jonathan Tellier

Shibboleth connector by Jonathan Tellier. The Shibboleth System is a standards based, open source software package for web single sign-on across or within organizational boundaries. It allows sites to make informed authorization decisions for individual access of protected online resources in a privacy-preserving manner

Griffon plugin, Apache 2 license

JCA connectors

• OPS4J Orient provides a JCA resource adapter for integrating OrientDB with Java EE 6 servers
• OrientDB JCA connector to access to OrientDB database via JCA API + XA Transactions

Pacer plugin by Paul Dlug. Pacer is a JRuby graph traversal framework built on the Tinkerpop stack. This plugin enables full OrientDB graph support in Pacer.

EventStore for Axonframework, which uses fully transactional (full ACID support) NoSQL database OrientDB. Axon Framework helps build scalable, extensible and maintainable applications by supporting developers apply the Command Query Responsibility Segregation (CQRS) architectural pattern

Accessing OrientDB using Slick

Jackrabbit module to use OrientDB as backend.

orientqb

orientqb is a builder for OSQL query language written in Java. orientqb has been thought to help developers in writing complex queries dynamically and aims to be simple but powerful.

Multi-Model

The OrientDB engine supports Graph, Document, Key/Value, and Object models, so you can use OrientDB as a replacement for a product in any of these categories. However, the main reason why users choose OrientDB is because of its true Multi-Model DBMS abilities, which combine all the features of the four models into the core. These abilities are not just interfaces to the database engine, but rather the engine itself was built to support all four models. This is also the main difference to other multi-model DBMSs, as they implement an additional layer with an API, which mimics additional models. However, under the hood, they're truly only one model, therefore they are limited in speed and scalability.

The Document Model

The data in this model is stored inside documents. A document is a set of key/value pairs (also referred to as fields or properties), where the key allows access to its value. Values can hold primitive data types, embedded documents, or arrays of other values. Documents are not typically forced to have a schema, which can be advantageous, because they remain flexible and easy to modify. Documents are stored in collections, enabling developers to group data as they decide. OrientDB uses the concepts of "classes" and "clusters" as its form of "collections" for grouping documents. This provides several benefits, which we will discuss in further sections of the documentation.

OrientDB's Document model also adds the concept of a "LINK" as a relationship between documents. With OrientDB, you can decide whether to embed documents or link to them directly. When you fetch a document, all the links are automatically resolved by OrientDB. This is a major difference to other Document Databases, like MongoDB or CouchDB, where the developer must handle any and all relationships between the documents herself.

The table below illustrates the comparison between the relational model, the document model, and the OrientDB document model:

The Graph Model

A graph represents a network-like structure consisting of Vertices (also known as Nodes) interconnected by Edges (also known as Arcs). OrientDB's graph model is represented by the concept of a property graph, which defines the following:

• Vertex - an entity that can be linked with other Vertices and has the following mandatory properties:

  • unique identifier
  • set of incoming Edges
  • set of outgoing Edges

• Edge - an entity that links two Vertices and has the following mandatory properties:

  • unique identifier
  • link to an incoming Vertex (also known as head)
  • link to an outgoing Vertex (also known as tail)
  • label that defines the type of connection/relationship between head and tail vertex

In addition to mandatory properties, each vertex or edge can also hold a set of custom properties. These properties can be defined by users, which can make vertices and edges appear similar to documents. In the table below, you can find a comparison between the graph model, the relational data model, and the OrientDB graph model:

The Key/Value Model

This is the simplest model of the three. Everything in the database can be reached by a key, where the values can be simple and complex types. OrientDB supports Documents and Graph Elements as values allowing for a richer model, than what you would normally find in the classic Key/Value model. The classic Key/Value model provides "buckets" to group key/value pairs in different containers. The most classic use cases of the Key/Value Model are:

• POST the value as payload of the HTTP call -> /<bucket>/<key>
• GET the value as payload from the HTTP call -> /<bucket>/<key>
• DELETE the value by Key, by calling the HTTP call -> /<bucket>/<key>

The table below illustrates the comparison between the relational model, the Key/Value model, and the OrientDB Key/Value model:

The Object Model

This model has been inherited by Object Oriented programming and supports Inheritance between types (sub-types extends the super-types), Polymorphism when you refer to a base class and Direct binding from/to Objects used in programming languages.

The table below illustrates the comparison between the relational model, the Object model, and the OrientDB Object model:

Multi Tenant

There are at many ways to build multi-tenant applications on top of OrientDB, in this page we are going to analyze pros and cons of three of the most used approaches.

One database per tenant

With this solution, each tenant is a database. The OrientDB server allows to host multiple databases.

Pros:

• Easy to use: to create/drop a new tenant, simply create/drop the database
• Easy to scale up by moving the database on different servers

Cons:

• Hard to create reports and analytics cross tenant, it requires to execute the same query against all the databases

Specific clusters per tenant

With this solution, each tenant is stored in one or more clusters of the same database. For example, the class Product could have the following clusters: Product_ClientA, Product_ClientB, Product_ClientC. In this way a query against a specific cluster will be used to retrieve data from one tenant only. Example to retrieve all the products of 2016, ordered by the most recent, only for the tenant "ClientC": select * from cluster:Product_ClientC where date >= '2016-01-01' order by date desc.

Instead, a query against the class Product will return a cross-tenant result. Example: select * from Product where date >= '2016-01-01' order by date desc.

Pros:

• Easy to create reports and analytics cross tenant, because it's just one database
• It's possible to scale up on multiple servers by using sharding

Cons:

• The maximum number of clusters per database is 32,768. To bypass this limitation, use multiple databases
• No security out of the box, it's entirely up to the application to isolate access between tenants

Use Partitioned Graphs

By using the OrientDB's record level security, it's possible to have multiple partitions of graphs accessible by different users. For more information look at Partitioned Graphs.

Pros:

• Easy to create reports and analytics cross tenant, because it's just one database

Cons:

• Performance: record level security has an overhead, specially with queries
• Scales worse than "One database per tenant" solution, because the database will end up to be much bigger
• Sharding is not possible because records of multiple tenants are mixed on the same clusters
• Hard to guarantee a predictable level of service for all the users, because users connect to a tenant impact the other tenants

Basic Concepts

Record

The smallest unit that you can load from and store in the database. Records come in four types:

• Documents
• Blobs
• Vertices
• Edges

A Record is the smallest unit that can be loaded from and stored into the database. A record can be a Document, a Blob a Vertex or even an Edge.

Document

The Document is the most flexible record type available in OrientDB. Documents are softly typed and are defined by schema classes with defined constraints, but you can also use them in a schema-less mode too.

Documents handle fields in a flexible manner. You can easily import and export them in JSON format. For example,

{
     "name"      : "Jay",
     "surname"   : "Miner",
     "job"       : "Developer",
     "creations" : [
          {
		       "name"    : "Amiga 1000",
               "company" : "Commodore Inc."
		   }, {
		       "name"    : "Amiga 500",
               "company" : "Commodore Inc."
           }
     ]
}

For Documents, OrientDB also supports complex relationships. From the perspective of developers, this can be understood as a persistent Map<String,Object>.

BLOB

In addition to the Document record type, OrientDB can also load and store binary data. The BLOB record type was called RecordBytes before OrientDB v2.2.

Vertex

In Graph databases, the most basic unit of data is the node, which in OrientDB is called a vertex. The Vertex stores information for the database. There is a separate record type called the Edge that connects one vertex to another.

Vertices are also documents. This means they can contain embedded records and arbitrary properties.

Edge

In Graph databases, an arc is the connection between two nodes, which in OrientDB is called an edge. Edges are bidirectional and can only connect two vertices.

Edges can be regular or lightweight. The Regular Edge saves as a Document, while the Lightweight Edge does not. For an understanding of the differences between these, see Lightweight Edges.

For more information on connecting vertices in general, see Relationships, below.

Record ID

When OrientDB generates a record, it auto-assigns a unique unit identifier, called a Record ID, or RID. The syntax for the Record ID is the pound sign with the cluster identifier and the position. The format is like this:

#<cluster>:<position>.

• Cluster Identifier: This number indicates the cluster to which the record belongs. Positive numbers in the cluster identifier indicate persistent records. Negative numbers indicate temporary records, such as those that appear in result-sets for queries that use projections.

• Position: This number defines the absolute position of the record in the cluster.

NOTE: The prefix character # is mandatory to recognize a Record ID.

Records never lose their identifiers unless they are deleted. When deleted, OrientDB never recycles identifiers. Additionally, you can access records directly through their Record ID's. For this reason, you don't need to create a field to serve as the primary key, as you do in Relational databases.

Record Version

Records maintain their own version number, which increments on each update. In optimistic transactions, OrientDB checks the version in order to avoid conflicts at commit time.

Class

The concept of the Class is taken from the Object Oriented Programming paradigm. In OrientDB, classes define records. It is closest to the concept of a table in Relational databases.

Classes can be schema-less, schema-full or a mix. They can inherit from other classes, creating a tree of classes. Inheritance, in this context, means that a sub-class extends a parent class, inheriting all of its attributes.

Each class has its own clusters (data files). A non-abstract class (see below) must have at least one cluster defined, which functions as its default cluster. But, a class can support multiple clusters. When you execute a query against a class, it automatically propagates to all clusters that are part of the class. When you create a new record, OrientDB selects the cluster to store it in using a configurable strategy.

When you create a new class, by default, OrientDB creates new persistent clusters with the same name as the class, in lowercase, suffixed with underscore and an integer. As a default, OrientDB creates as many clusters per class as many cores (processors) the host machine has.

Eg. for class Person, OrientDB will create clusters person, person_1, person_2 and so on so forth.

Abstract Class

The concept of an Abstract Class is one familiar to Object-Oriented programming. In OrientDB, this feature has been available since version 1.2.0. Abstract classes are classes used as the foundation for defining other classes. They are also classes that cannot have instances. For more information on how to create an abstract class, see CREATE CLASS.

This concept is essential to Object Orientation, without the typical spamming of the database with always empty, auto-created clusters.

For more information on Abstract Class as a concept, see Abstract Type and Abstract Methods and Classes

Class vs. Cluster in Queries

The combination of classes and clusters is very powerful and has a number of use cases. Consider an example where you create a class Invoice, with two clusters invoice2015 and invoice2016. You can query all invoices using the class as a target with SELECT.

orientdb> SELECT FROM Invoice

In addition to this, you can filter the result-set by year. The class Invoice includes a year field, you can filter it through the WHERE clause.

orientdb> SELECT FROM Invoice WHERE year = 2012

You can also query specific objects from a single cluster. By splitting the class Invoice across multiple clusters, (that is, one per year), you can optimize the query by narrowing the potential result-set.

orientdb> SELECT FROM CLUSTER:invoice2012

Due to the optimization, this query runs significantly faster, because OrientDB can narrow the search to the targeted cluster.

Cluster

Where classes provide you with a logical framework for organizing data, clusters provide physical or in-memory space in which OrientDB actually stores the data. It is comparable to the collection in Document databases and the table in Relational databases.

When you create a new class, the CREATE CLASS process also creates physical clusters that serve as the default location in which to store data for that class. OrientDB forms the cluster names using the class name, with all lower case letters. Beginning with version 2.2, OrientDB creates additional clusters for each class, (one for each CPU core on the server), to improve performance of parallelism.

For more information, see the Clusters Tutorial.

Materialized View

A materialized view is a persistent object that contains the result of a query. In terms of SQL querying, it can be considered as the equivalent of a class, that means that it can be used as a target for queries.

A materialized view can be configured to be read-only or updatable. Updating a record of a materialized view results in the update of the original record (ie. the record from which the view raw was created).

Views can have indexes, like normal classes.

Relationships

OrientDB supports two kinds of relationships: referenced and embedded. It can manage relationships in a schema-full or schema-less scenario.

Referenced Relationships

In Relational databases, tables are linked through JOIN commands, which can prove costly on computing resources. OrientDB manges relationships natively without computing JOIN's. Instead, it stores direct links to the target objects of the relationship. This boosts the load speed for the entire graph of connected objects, such as in Graph and Object database systems.

For example

                  customer
  Record A     ------------->    Record B
CLASS=Invoice                 CLASS=Customer
  RID=5:23                       RID=10:2

Here, record A contains the reference to record B in the property customer. Note that both records are reachable by other records, given that they have a Record ID.

With the Graph API, Edges are represented with two links stored on both vertices to handle the bidirectional relationship.

1:1 and 1:n Referenced Relationships

OrientDB expresses relationships of these kinds using links of the LINK type.

1:n and n:n Referenced Relationships

OrientDB expresses relationships of these kinds using a collection of links, such as:

• LINKLIST An ordered list of links.
• LINKSET An unordered set of links, which does not accept duplicates.
• LINKMAP An ordered map of links, with String as the key type. Duplicates keys are not accepted.

With the Graph API, Edges connect only two vertices. This means that 1:n relationships are not allowed. To specify a 1:n relationship with graphs, create multiple edges.

Embedded Relationships

When using Embedded relationships, OrientDB stores the relationship within the record that embeds it. These relationships are stronger than Reference relationships. You can represent it as a UML Composition relationship.

Embedded records do not have their own Record ID, given that you can't directly reference it through other records. It is only accessible through the container record.

In the event that you delete the container record, the embedded record is also deleted. For example,

                   address
  Record A     <>---------->   Record B
CLASS=Account               CLASS=Address
  RID=5:23                     NO RID!

Here, record A contains the entirety of record B in the property address. You can reach record B only by traversing the container record. For example,

orientdb> SELECT FROM Account WHERE address.city = 'Rome'

1:1 and n:1 Embedded Relationships

OrientDB expresses relationships of these kinds using the EMBEDDED type.

1:n and n:n Embedded Relationships

OrientDB expresses relationships of these kinds using a collection of links, such as:

• EMBEDDEDLIST An ordered list of records.
• EMBEDDEDSET An unordered set of records, that doesn't accept duplicates.
• EMBEDDEDMAP An ordered map of records as the value and a string as the key, it doesn't accept duplicate keys.

Inverse Relationships

In OrientDB, all Edges in the Graph model are bidirectional. This differs from the Document model, where relationships are always unidirectional, requiring the developer to maintain data integrity. In addition, OrientDB automatically maintains the consistency of all bidirectional relationships.

Database

The database is an interface to access the real Storage. IT understands high-level concepts such as queries, schemas, metadata, indices and so on. OrientDB also provides multiple database types. For more information on these types, see Database Types.

Each server or Java VM can handle multiple database instances, but the database name must be unique. You can't manage two databases at the same time, even if they are in different directories. To handle this case, use the $ dollar character as a separator instead of the / slash character. OrientDB binds the entire name, so it becomes unique, but at the file system level it converts $ with /, allowing multiple databases with the same name in different paths. For example,

test$customers -> test/customers
production$customers = production/customers

Database URL

OrientDB uses its own URL format, of engine and database name as <engine>:<db-name>.

Database Usage

You must always close the database once you finish working on it.

NOTE: OrientDB automatically closes all opened databases, when the process dies gracefully (not by killing it by force). This is assured if the Operating System allows a graceful shutdown.

OrientDB Programs

When you install OrientDB on your system, there are a number of applications included in the package. You can find them in the $ORIENTDB_HOME/bin/ directory. These applications include the OrientDB Server, the Console used to connect to the server and various other programs and administrative utilities.

OrientDB Server

OrientDB provides several methods of starting and managing the Server process. These include individual scripts that initialize the Java Virtual Machine and start OServer and systemd and initscripts that allow you to launch and manage the Server using the systemctl or service utilities on Linux.

Programs

There are two scripts that you can use to start the OrientDB Server directly. Each script has two builds: a .sh shell script for use on Linux distributions and a .bat batch file for use on Windows.

Process Management

Where the above programs make it easy to start the Server on your desktop for experimentation and similar tasks, it is not very convenient for use on a production host. In addition to the Server startup scripts, OrientDB also ships with a systemd service file and an initscript, which you can install on Linux distributions to start, stop and restart the OrientDB Server using either systemctl or service utilities.

For more information, see OrientDB Server Process.

OrientDB Server - server

The OrientDB Server is a Java application that manages database operations, retrieval and storage. The shell script server provides a convenient command-line interface to use in configuring and launching the OrientDB Server.

OrientDB provides a server.sh shell script for starting the server on Linux operating systems and a server.bat batch file for starting it on Windows.

Configuration Environmental Variables

OrientDB manages most of its configuration options through various XML configuration files. However, there are a handful of environmental variables that it utilizes when starting the Server. When the script runs, it check these variables to determine whether any value has been set on them. In the event that it finds none, it instead sets a sensible default value.

On Linux, you can set the environmental variable using the export command:

$ export ORIENTDB_HOME=/opt/orientdb/3.2.36/

The following environmental variables are used by server.sh:

The server.sh script also accepts argument. If one of the arguments is debug, the script enables the following Java options -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=1044. Any additional arguments are passed directly to the java program.

OrientDB Server - dserver.sh

In situations where an individual host is insufficient for your storage or performance needs, you can run OrientDB on multiple hosts distributing the load between them. Where the server script starts OrientDB as an individual Server, the dserver script does the same, while configuring Server to operate in a distributed environment.

OrientDB provides a dserver.sh shell script for Linux operating systems and dserver.bat batch file for the Windows operating system.

Configuration Environmental Variables

OrientDB manages most of its configuration options through various XML configuration files. However, there are a handful of environmental variables that it utilizes when starting the Server. When the script runs, it check these variables to determine whether any value has been set on them. In the event that it finds none, it instead sets a sensible default value.

On Linux, you can set the environmental variable using the export command:

$ export ORIENTDB_HOME=/opt/orientdb/3.2.36/

The following environmental variables are used by server.sh:

The dserver.sh script also accepts argument. If one of the arguments is debug, the script enables the following Java options -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=1044. Any additional arguments are passed directly to the java program.

OrientDB Server Process

Using the server or dserver shell scripts or batch files you can manually start the OrientDB Server. While this is often sufficient for testing or experimentation, however, in production environments it is preferable to user process and service management tools, like init and systemd.

Service Installation

In order to manage OrientDB through process or service management tools, you first need to install it on your system. The bin/ directory of your OrientDB installation contains an orientdb.sh script that you need to modify and install at /etc/init.d/orientdb.

For instance,

# cp $ORIENTDB_HOME/bin/orientdb.sh /etc/init.d/orientdb

Once you have the file in place, edit it and modify the two script variables at the top of the file.

Service File

In cases where your host uses systemd for service and process management, you also need to copy the orientdb.service file.

# cp $ORIENTDB_HOME/bin/orientdb.service /etc/systemd/system

Once this is done, edit the service file to update values on the following fields:

Usage

On operating systems that use init for process management, copying the OrientDB script and setting the variables is enough to manage it using the service command. On hosts that utilize systemd, you also need to copy over the orientdb.service file.

Usage with init

When working with a host that uses init, you can manage the OrientDB Server using the following commands:

To start the OrientDB Server:

# service orientdb start

To stop the OrientDB Server

# service orientdb stop

To start OrientDB when the server boots:

# service orientdb enable

Usage with systemd

When working with a host that uses systemd, you can manage the OrientDB Server using the following commands:

To start the OrientDB Server:

# systemctl start orientdb 

To stop the OrientDB Server

# systemctl stop orientdb 

To start OrientDB when the server boots:

# systemctl enable orientdb 

OrientDB Client Programs

OrientDB ships with one client utility: the Console. The Console is an interactive text-based interface for administering OrientDB Server and databases.

Console

OrientDB provides a Console Tool, which is a Java application that connects to and operates on OrientDB databases and Server instances.

Console Modes

There are two modes available to you, while executing commands through the OrientDB Console: interactive mode and batch mode.

Interactive Mode

By default, the Console starts in interactive mode. In this mode, the Console loads to an orientdb> prompt. From there you can execute commands and SQL statements as you might expect in any other database console.

You can launch the console in interactive mode by executing the console.sh for Linux OS systems or console.bat for Windows systems in the bin directory of your OrientDB installation. Note that running this file requires execution permissions.

$ cd $ORIENTDB_HOME/bin
$ ./console.sh

OrientDB console v.X.X.X (build 0) www.orientdb.com
Type 'HELP' to display all the commands supported.
Installing extensions for GREMLIN language v.X.X.X

orientdb>

From here, you can begin running SQL statements or commands. For a list of these commands, see commands.

Batch mode

When the Console runs in batch mode, it takes commands as arguments on the command-line or as a text file and executes the commands in that file in order. Use the same console.sh or console.bat file found in bin at the OrientDB installation directory.

• Command-line: To execute commands in batch mode from the command line, pass the commands you want to run in a string, separated by a semicolon.

  $ $ORIENTDB_HOME/bin/console.sh "CONNECT REMOTE:localhost/demo;SELECT FROM Profile"
  

• Script Commands: In addition to entering the commands as a string on the command-line, you can also save the commands to a text file as a semicolon-separated list.

  $ vim commands.txt
  
    CONNECT REMOTE:localhost/demo;SELECT FROM Profile
  
  $ $ORIENTDB_HOME/bin/console.sh commands.txt
  

Ignoring Errors

When running commands in batch mode, you can tell the console to ignore errors, allowing the script to continue the execution, with the ignoreErrors setting.

$ vim commands.txt

  SET ignoreErrors TRUE

Enabling Echo

Regardless of whether you call the commands as an argument or through a file, when you run console commands in batch mode, you may also need to display them as they execute. You can enable this feature using the echo setting, near the start of your commands list.

$ vim commands.txt

  SET echo TRUE

Enabling Date in prompt

Starting from v2.2.9, to enable the date in the prompt, set the variable promptDateFormat with the date format following the SimpleDateFormat specs.

orientdb {db=test1}> set promptDateFormat "yyy-MM-dd hh:mm:ss.sss"

orientdb {db=test1 (2016-08-26 09:34:12.012)}> 

Console Commands

OrientDB implements a number of SQL statements and commands that are available through the Console. In the event that you need information while working in the console, you can access it using either the HELP or ? command.