download.arangodb.com€¦ · 1.1 1.2 1.2.1 1.2.1.1 1.2.1.2 1.2.1.3 1.2.1.4 1.2.2 1.2.3 1.2.4 1.3...

1.2.1.1

1.2.1.2

1.2.1.3

1.2.1.4

1.4.4.1

1.4.4.2

1.4.4.3

1.5.2.1

1.5.2.2

1.5.3.1

1.5.3.2

1.5.4.1

1.5.4.2

1.5.4.3

1.5.6.1

1.5.6.2

1.5.7.1

1.5.7.2

1.5.7.3

1.5.7.4

TableofContentsIntroduction

GettingStarted

Installing

MacOSX

Windows

Compiling

Authentication

AccessingtheWebInterface

ComingfromSQL

Highlights

Scalability

Architecture

Datamodels

Limitations

Datacentertodatacenterreplication

Introduction

Applicability

Requirements

Datamodels&modeling

Concepts

Databases

WorkingwithDatabases

NotesaboutDatabases

Collections

CollectionMethods

DatabaseMethods

Documents

BasicsandTerminology

CollectionMethods

DatabaseMethods

Graphs,Vertices&Edges

DatabaseMethods

ViewMethods

NamingConventions

DatabaseNames

CollectionNames

DocumentKeys

AttributeNames

Indexing

1.6.4.1

1.6.4.2

1.6.4.3

1.6.4.4

1.6.4.5

1.6.4.6

1.7.1.1

1.7.1.2

1.7.2.1

1.7.3.1

1.7.3.2

1.8.1.1

1.9.7.1

1.9.7.2

1.9.7.3

1.9.7.4

1.9.9.1

1.9.9.1.1

1.9.9.1.2

1.9.9.2

1.9.9.2.1

1.9.9.2.2

IndexBasics

Whichindextousewhen

IndexUtilization

WorkingwithIndexes

HashIndexes

Skiplists

Persistent

FulltextIndexes

GeoIndexes

VertexCentricIndexes

Graphs

GeneralGraphs

GraphManagement

GraphFunctions

SmartGraphs

SmartGraphManagement

Traversals

UsingTraversalObjects

ExampleData

WorkingwithEdges

Pregel

ArangoSearch

Analyzers

FoxxMicroservices

Ataglance

Gettingstarted

Servicemanifest

Servicecontext

Configuration

Dependencies

Routers

Endpoints

Middleware

Request

Response

UsingGraphQL

Sessionsmiddleware

Sessionstorages

Collectionstorage

JWTstorage

Sessiontransports

Cookietransport

Headertransport

1.9.10

1.9.11

1.9.12

1.9.13

1.9.14

1.9.14.1

1.9.14.2

1.9.14.3

1.9.14.4

1.9.14.5

1.9.14.5.1

1.9.14.5.2

1.9.14.5.3

1.9.14.5.4

1.9.14.5.5

1.9.14.5.6

1.9.14.6

1.9.14.7

1.9.14.8

1.9.15

1.9.16

1.9.17

1.9.17.1

1.9.17.2

1.9.17.3

1.10.1

1.10.2

1.10.3

1.10.4

1.10.5

1.11.1

1.11.2

1.11.2.1

1.11.2.2

1.11.2.3

1.11.2.3.1

1.11.2.3.2

1.11.2.3.3

1.11.2.3.4

1.11.3

1.11.3.1

1.11.3.2

Servingfiles

Writingtests

CrossOrigin

Scriptsandqueuedjobs

Migrating2.xservices

Migratingfrompre-2.8

manifest.json

applicationContext

RepositoriesandModels

Controllers

Requestcontext

Errorhandling

Before/After/Around

Requestobject

Responseobject

DependencyInjection

Sessions

AuthandOAuth2

FoxxQueries

Legacycompatibilitymode

Usermanagement

Relatedmodules

Authentication

OAuth1.0a

OAuth2.0

Transactions

Transactioninvocation

Passingparameters

Lockingandisolation

Durability

Limitations

Deployment

Singleinstance

Cluster

Cluster:Mesos,DC/OS

Cluster:Generic&Docker

AdvancedTopics

StandaloneAgency

Cluster:Localtestsetups

Cluster:Processes

Cluster:Docker

MultipleDatacenters

Cluster

Kafka&Zookeeper

1.11.3.3

1.11.3.4

1.11.3.5

1.12.1

1.12.1.1

1.12.1.2

1.12.1.3

1.12.1.4

1.12.1.5

1.12.1.6

1.12.1.7

1.12.1.8

1.12.1.9

1.12.2

1.12.2.1

1.12.2.2

1.12.2.3

1.12.3

1.12.4

1.12.5

1.12.6

1.12.7

1.12.7.1

1.12.8

1.12.8.1

1.12.8.2

1.12.8.3

1.12.8.4

1.12.8.5

1.12.8.6

1.12.8.7

1.12.8.8

1.12.8.9

1.12.8.10

1.12.8.11

1.12.8.12

1.12.9

1.12.10

1.12.11

1.12.11.1

1.12.11.2

1.12.12

1.12.12.1

ArangoSyncMaster

ArangoSyncWorkers

Prometheus&Grafana

Administration

WebInterface

Dashboard

Cluster

Collections

Document

Queries

Graphs

Services

ArangoDBShell

ShellOutput

Configuration

Details

Arangoimport

Arangodump

Arangorestore

Arangoexport

ManagingUsers

InArangosh

ServerConfiguration

OperatingSystemConfiguration

ManagingEndpoints

SSLConfiguration

LDAPOptions

LoggingOptions

GeneralOptions

Write-AheadLogOptions

CompactionOptions

ClusterOptions

RocksDBEngineOptions

HashCacheOptions

AsynchronousTasks

Durability

Encryption

Auditing

Configuration

Events

Replication

AsynchronousReplication

1.12.12.1.1

1.12.12.1.2

1.12.12.1.3

1.12.12.1.4

1.12.12.1.5

1.12.12.2

1.12.12.2.1

1.12.12.2.2

1.12.12.3

1.12.13

1.12.14

1.12.15

1.12.15.1

1.12.15.2

1.12.15.3

1.12.15.4

1.12.15.5

1.12.15.6

1.12.15.7

1.12.15.8

1.12.15.9

1.12.15.10

1.13.1

1.13.2

1.13.3

1.13.4

1.13.5

1.14.1

1.15.1

1.16.1

1.16.2

1.17.1

1.17.2

1.17.3

1.17.4

1.17.5

1.17.6

1.17.7

1.17.8

Components

Per-DatabaseSetup

Server-LevelSetup

SyncingCollections

ReplicationLimitations

SynchronousReplication

Implementation

Configuration

SatelliteCollections

Sharding

Upgrading

Upgradingto3.3

Upgradingto3.2

Upgradingto3.1

Upgradingto3.0

Upgradingto2.8

Upgradingto2.6

Upgradingto2.5

Upgradingto2.4

Upgradingto2.3

Upgradingto2.2

Troubleshooting

arangod

EmergencyConsole

DatafileDebugger

Arangobench

Monitoring

Security

Architecture

Write-aheadlog

StorageEngines

Releasenotes

WhatsNewin3.3

Incompatiblechangesin3.3

WhatsNewin3.2

KnownIssuesin3.2

WhatsNewin3.1

WhatsNewin3.0

1.17.9

1.17.10

1.17.11

1.17.12

1.17.13

1.17.14

1.17.15

1.17.16

1.17.17

1.17.18

1.17.19

1.17.20

1.17.21

1.17.22

1.17.23

1.18.1

1.18.1.1

1.18.1.2

1.18.1.3

1.18.2

1.18.2.1

1.18.2.2

1.18.2.3

1.18.2.4

1.18.2.5

1.18.2.6

1.18.2.7

1.18.2.8

1.18.2.9

1.18.3

1.18.3.1

1.18.3.1.1

1.18.3.1.2

1.18.3.1.3

1.18.3.1.4

1.18.3.2

1.18.3.2.1

1.18.3.2.2

1.18.3.2.3

1.18.4

1.18.5

WhatsNewin2.8

WhatsNewin2.7

WhatsNewin2.6

WhatsNewin2.5

WhatsNewin2.4

WhatsNewin2.3

WhatsNewin2.2

WhatsNewin2.1

Appendix

References

collection

cursor

JavaScriptModules

@arangodb

console

crypto

request

actions

queries

Write-aheadlog

TaskManagement

Deprecated

SimpleQueries

Pagination

ModificationQueries

GeoQueries

FulltextQueries

Actions

DeliveringHTMLPages

JsonObjects

Modifying

Errorcodesandmeanings

Glossary

ArangoDBv3.4.milestone0DocumentationWelcometotheArangoDBdocumentation!

NewandeagertotryoutArangoDB?Startrightawaywithourbeginner'sguide:GettingStarted

Thedocumentationisorganizedinfourhandbooks:

ThismanualdescribesArangoDBanditsfeaturesindetailforyouasauser,developerandadministrator.TheAQLhandbookexplainsArangoDB'squerylanguageAQL.TheHTTPhandbookdescribestheinternalAPIofArangoDBthatisusedtocommunicatewithclients.Ingeneral,theHTTPhandbookwillbeofinteresttodriverdevelopers.Ifyouuseanyoftheexistingdriversforthelanguageofyourchoice,youcanskipthishandbook.Ourcookbookwithrecipesforspecificproblemsandsolutions.

Featuresareillustratedwithinteractiveusageexamples;youcancut'n'pastethemintoarangoshtotrythemout.TheHTTPREST-APIfordriverdevelopersisdemonstratedwithcut'n'pasterecepiesintendedtobeusedwiththecURL.Driversmayprovidetheirownexamplesbasedonthese.jsbasedexamplestoimproveunderstandeabilityfortheirrespectiveusers,i.e.forthejavadriversomeofthesamplesarere-implemented.

Overview

ArangoDBisanativemulti-model,open-sourcedatabasewithflexibledatamodelsfordocuments,graphs,andkey-values.BuildhighperformanceapplicationsusingaconvenientSQL-likequerylanguageorJavaScriptextensions.UseACIDtransactionsifyourequirethem.Scalehorizontallyandverticallywithafewmouseclicks.

Keyfeaturesinclude:

installingArangoDBonaclusterisaseasyasinstallinganapponyourmobileFlexibledatamodeling:modelyourdataascombinationofkey-valuepairs,documentsorgraphs-perfectforsocialrelationsPowerfulquerylanguage(AQL)toretrieveandmodifydataUseArangoDBasanapplicationserverandfuseyourapplicationanddatabasetogetherformaximalthroughputTransactions:runqueriesonmultipledocumentsorcollectionswithoptionaltransactionalconsistencyandisolationReplicationandSharding:setupthedatabaseinamaster-slaveconfigurationorspreadbiggerdatasetsacrossmultipleserversConfigurabledurability:lettheapplicationdecideifitneedsmoredurabilityormoreperformanceNo-nonsensestorage:ArangoDBusesallofthepowerofmodernstoragehardware,likeSSDandlargecachesJavaScriptforall:nolanguagezoo,youcanuseonelanguagefromyourbrowsertoyourback-endArangoDBcanbeeasilydeployedasafault-tolerantdistributedstatemachine,whichcanserveastheanimalbrainofdistributedappliancesItisopensource(ApacheLicense2.0)

Community

IfyouhavequestionsregardingArangoDB,Foxx,drivers,orthisdocumentationdon'thesitatetocontactuson:

GitHubforissuesandmisbehaviororpullrequestsGoogleGroupsfordiscussionsaboutArangoDBingeneralortoannounceyournewFoxxAppStackOverflowforquestionsaboutAQL,usagescenariosetc.Slack,ourcommunitychat

Whenreportingissues,pleasedescribe:

theenvironmentyourunArangoDBintheArangoDBversionyouusewhetheryou'reusingFoxxtheclientyou'reusing

Introduction

whichpartsofthedocumentationyou'reworkingwith(link)whatyouexpecttohappenwhatisactuallyhappening

Wewillrespondassoonaspossible.

Introduction

Gettingstarted

Overview

Thisbeginner'sguidewillmakeyoufamiliarwithArangoDB.Wewillcoverhowto

installandrunalocalArangoDBserverusethewebinterfacetointeractwithitstoreexampledatainthedatabasequerythedatabasetoretrievethedataagaineditandremoveexistingdata

Installation

Headtoarangodb.com/download,selectyouroperatingsystemanddownloadArangoDB.Youmayalsofollowtheinstructionsonhowtoinstallwithapackagemanager,ifavailable.

IfyouinstalledabinarypackageunderLinux,theserverisautomaticallystarted.

IfyouinstalledArangoDBusinghomebrewunderMacOSX,starttheserverbyrunning/usr/local/sbin/arangod.

IfyouinstalledArangoDBunderWindowsasaservice,theserverisautomaticallystarted.Otherwise,runthearangod.exelocatedintheinstallationfolder'sbindirectory.YoumayhavetorunitasadministratortograntitwritepermissionstoC:\ProgramFiles.

Formorein-depthinformationonhowtoinstallArangoDB,aswellasavailablestartupparameters,installationinaclusterandsoon,seeInstalling.

ArangoDBofferstwostorageengines:MMFilesandRocksDB.Choosetheonewhichsuitsyourneedsbestintheinstallationprocessoronfirststartup.

Securingtheinstallation

Thedefaultinstallationcontainsonedatabase_systemandausernamedroot.

DebianbasedpackagesandtheWindowsinstallerwillaskforapasswordduringtheinstallationprocess.Red-Hatbasedpackageswillsetarandompassword.Forallotherinstallationpackagesyouneedtoexecute

shell>arango-secure-installation

Thiswillaskedforarootpasswordandsetsthispassword.

Webinterface

Theserveritself(arangod)speaksHTTP/REST,butyoucanusethegraphicalwebinterfacetokeepitsimple.There'salsoarangosh,asynchronousshellforinteractionwiththeserver.Ifyou'readeveloper,youmightprefertheshellovertheGUI.Itdoesnotprovidefeatureslikesyntaxhighlightinghowever.

WhenyoustartusingArangoDBinyourproject,youwilllikelyuseanofficialorcommunity-madedriverwritteninthesamelanguageasyourproject.Driversimplementaprogramminginterfacethatshouldfeelnaturalforthatprogramminglanguage,anddoallthetalkingtotheserver.Therefore,youcanmostcertainlyignoretheHTTPAPIunlessyouwanttowriteadriveryourselforexplicitlywanttousetherawinterface.

Togetfamiliarwiththedatabasesystemyoucanevenputdriversasideandusethewebinterface(codenameAardvark)forbasicinteraction.Thewebinterfacewillbecomeavailableshortlyafteryoustartedarangod.Youcanaccessitinyourbrowserathttp://localhost:8529-ifnot,pleaseseeTroubleshooting.

GettingStarted

Bydefault,authenticationisenabled.Thedefaultuserisroot.Dependingontheinstallationmethodused,theinstallationprocesseitherpromptedfortherootpasswordorthedefaultrootpasswordisempty(seeabove).

Nextyouwillbeaskedwhichdatabasetouse.Everyserverinstancecomeswitha_systemdatabase.Selectthisdatabasetocontinue.

GettingStarted

Youshouldthenbepresentedthedashboardwithserverstatisticslikethis:

Foramoredetaileddescriptionoftheinterface,seeWebInterface.

Databases,collectionsanddocuments

Databasesaresetsofcollections.Collectionsstorerecords,whicharereferredtoasdocuments.CollectionsaretheequivalentoftablesinRDBMS,anddocumentscanbethoughtofasrowsinatable.Thedifferenceisthatyoudon'tdefinewhatcolumns(orratherattributes)therewillbeinadvance.Everydocumentinanycollectioncanhavearbitraryattributekeysandvalues.Documentsinasinglecollectionwilllikelyhaveasimilarstructureinpracticehowever,butthedatabasesystemitselfdoesnotimposeitandwilloperatestableandfastnomatterhowyourdatalookslike.

Readmoreinthedata-modelconceptschapter.

Fornow,youcanstickwiththedefault_systemdatabaseandusethewebinterfacetocreatecollectionsanddocuments.StartbyclickingtheCOLLECTIONSmenuentry,thentheAddCollectiontile.Giveitaname,e.g.users,leavetheothersettingsunchanged(wewantittobeadocumentcollection)andSaveit.Anewtilelabeledusersshouldshowup,whichyoucanclicktoopen.

TherewillbeNodocumentsyet.Clickthegreencirclewiththewhiteplusontheright-handsidetocreateafirstdocumentinthiscollection.Adialogwillaskyoufora_key.YoucanleavethefieldblankandclickCreatetoletthedatabasesystemassignanautomaticallygenerated(unique)key.Notethatthe_keypropertyisimmutable,whichmeansyoucannotchangeitoncethedocumentiscreated.Whatyoucanuseasdocumentkeyisdescribedinthenamingconventions.

Anautomaticallygeneratedkeycouldbe"9883"(_keyisalwaysastring!),andthedocument_idwouldbe"users/9883"inthatcase.Asidefromafewsystemattributes,thereisnothinginthisdocumentyet.Let'saddacustomattributebyclickingtheicontotheleftof(emptyobject),thenAppend.Twoinputfieldswillbecomeavailable,FIELD(attributekey)andVALUE(attributevalue).Typenameaskeyandyournameasvalue.Appendanotherattribute,nameitageandsetittoyourage.ClickSavetopersistthechanges.IfyouclickonCollection:usersatthetopontheright-handsideoftheArangoDBlogo,thedocumentbrowserwillshowthedocumentsintheuserscollectionandyouwillseethedocumentyoujustcreatedinthelist.

Queryingthedatabase

GettingStarted

TimetoretrieveourdocumentusingAQL,ArangoDB'squerylanguage.Wecandirectlylookupthedocumentwecreatedviathe_id,buttherearealsootheroptions.ClicktheQUERIESmenuentrytobringupthequeryeditorandtypethefollowing(adjustthedocumentIDtomatchyourdocument):

RETURNDOCUMENT("users/9883")

ThenclickExecutetorunthequery.Theresultappearsbelowthequeryeditor:

"_key":"9883",

"_id":"users/9883",

"_rev":"9883",

"age":32,

"name":"JohnSmith"

Asyoucansee,theentiredocumentincludingthesystemattributesisreturned.DOCUMENT()isafunctiontoretrieveasingledocumentoralistofdocumentsofwhichyouknowthe_keysor_ids.Wereturntheresultofthefunctioncallasourqueryresult,whichisourdocumentinsideoftheresultarray(wecouldhavereturnedmorethanoneresultwithadifferentquery,butevenforasingledocumentasresult,westillgetanarrayatthetoplevel).

Thistypeofqueryiscalleddataaccessquery.Nodataiscreated,changedordeleted.Thereisanothertypeofquerycalleddatamodificationquery.Let'sinsertaseconddocumentusingamodificationquery:

INSERT{name:"KatieFoster",age:27}INTOusers

Thequeryisprettyself-explanatory:theINSERTkeywordtellsArangoDBthatwewanttoinsertsomething.Whattoinsert,adocumentwithtwoattributesinthiscase,followsnext.Thecurlybraces{}signifydocuments,orobjects.Whentalkingaboutrecordsinacollection,wecallthemdocuments.EncodedasJSON,wecallthemobjects.Objectscanalsobenested.Here'sanexample:

"name":{

"first":"Katie",

"last":"Foster"

INTOisamandatorypartofeveryINSERToperationandisfollowedbythecollectionnamethatwewanttostorethedocumentin.Notethattherearenoquotemarksaroundthecollectionname.

Ifyourunabovequery,therewillbeanemptyarrayasresultbecausewedidnotspecifywhattoreturnusingaRETURNkeyword.Itisoptionalinmodificationqueries,butmandatoryindataaccessqueries.EvenwithRETURN,thereturnvaluecanstillbeanemptyarray,e.g.ifthespecifieddocumentwasnotfound.Despitetheemptyresult,theabovequerystillcreatedanewuserdocument.Youcanverifythiswiththedocumentbrowser.

Let'saddanotheruser,butreturnthenewlycreateddocumentthistime:

INSERT{name:"JamesHendrix",age:69}INTOusers

RETURNNEW

NEWisapseudo-variable,whichreferstothedocumentcreatedbyINSERT.Theresultofthequerywilllooklikethis:

"_key":"10074",

"_id":"users/10074",

"_rev":"10074",

"age":69,

"name":"JamesHendrix"

GettingStarted

Nowthatwehave3usersinourcollection,howtoretrievethemallwithasinglequery?Thefollowingdoesnotwork:

TherecanonlybeasingleRETURNstatementhereandasyntaxerrorisraisedifyoutrytoexecuteit.TheDOCUMENT()functionoffersasecondarysignaturetospecifymultipledocumenthandles,sowecoulddo:

RETURNDOCUMENT(["users/9883","users/9915","users/10074"])

Anarraywiththe_idsofall3documentsispassedtothefunction.Arraysaredenotedbysquarebrackets[]andtheirelementsareseparatedbycommas.

Butwhatifweaddmoreusers?Wewouldhavetochangethequerytoretrievethenewlyaddedusersaswell.Allwewanttosaywithourqueryis:"Foreveryuserinthecollectionusers,returntheuserdocument".WecanformulatethiswithaFORloop:

FORuserINusers

RETURNuser

Itexpressestoiterateovereverydocumentinusersandtouseuserasvariablename,whichwecanusetorefertothecurrentuserdocument.Itcouldalsobecalleddoc,uorahuacatlguacamole,thisisuptoyou.Itisadvisabletouseashortandself-descriptivenamehowever.

Theloopbodytellsthesystemtoreturnthevalueofthevariableuser,whichisasingleuserdocument.Alluserdocumentsarereturnedthisway:

"_key":"9915",

"_id":"users/9915",

"_rev":"9915",

"age":27,

"name":"KatieFoster"

"_key":"9883",

"_id":"users/9883",

"_rev":"9883",

"age":32,

"name":"JohnSmith"

"_key":"10074",

"_id":"users/10074",

"_rev":"10074",

"age":69,

"name":"JamesHendrix"

Youmayhavenoticedthattheorderofthereturneddocumentsisnotnecessarilythesameastheywereinserted.Thereisnoorderguaranteedunlessyouexplicitlysortthem.WecanaddaSORToperationveryeasily:

FORuserINusers

SORTuser._key

RETURNuser

Thisdoesstillnotreturnthedesiredresult:James(10074)isreturnedbeforeJohn(9883)andKatie(9915).Thereasonisthatthe_keyattributeisastringinArangoDB,andnotanumber.Theindividualcharactersofthestringsarecompared.1islowerthan9andtheresultistherefore"correct".Ifwewantedtousethenumericalvalueofthe_keyattributesinstead,wecouldconvertthestringtoanumberanduseitforsorting.Therearesomeimplicationshowever.Wearebetteroffsortingsomethingelse.Howabouttheage,indescendingorder?

GettingStarted

FORuserINusers

SORTuser.ageDESC

RETURNuser

Theuserswillbereturnedinthefollowingorder:James(69),John(32),Katie(27).InsteadofDESCfordescendingorder,ASCcanbeusedforascendingorder.ASCisthedefaultthoughandcanbeomitted.

Wemightwanttolimittheresultsettoasubsetofusers,basedontheageattributeforexample.Let'sreturnusersolderthan30only:

FORuserINusers

FILTERuser.age>30

SORTuser.age

RETURNuser

ThiswillreturnJohnandJames(inthisorder).Katie'sageattributedoesnotfulfillthecriterion(greaterthan30),sheisonly27andthereforenotpartoftheresultset.Wecanmakeheragetoreturnheruserdocumentagain,usingamodificationquery:

UPDATE"9915"WITH{age:40}INusers

RETURNNEW

UPDATEallowstopartiallyeditanexistingdocument.ThereisalsoREPLACE,whichwouldremoveallattributes(exceptfor_keyand_id,whichremainthesame)andonlyaddthespecifiedones.UPDATEontheotherhandonlyreplacesthespecifiedattributesandkeepseverythingelseas-is.

TheUPDATEkeywordisfollowedbythedocumentkey(oradocument/objectwitha_keyattribute)toidentifywhattomodify.TheattributestoupdatearewrittenasobjectaftertheWITHkeyword.INdenotesinwhichcollectiontoperformthisoperationin,justlikeINTO(bothkeywordsareactuallyinterchangablehere).ThefulldocumentwiththechangesappliedisreturnedifweusetheNEWpseudo-variable:

"_key":"9915",

"_id":"users/9915",

"_rev":"12864",

"age":40,

"name":"KatieFoster"

IfweusedREPLACEinstead,thenameattributewouldbegone.WithUPDATE,theattributeiskept(thesamewouldapplytoadditionalattributesifwehadthem).

LetusrunourFILTERqueryagain,butonlyreturntheusernamesthistime:

FORuserINusers

FILTERuser.age>30

SORTuser.age

RETURNuser.name

Thiswillreturnthenamesofall3users:

"JohnSmith",

"KatieFoster",

"JamesHendrix"

Itiscalledaprojectionifonlyasubsetofattributesisreturned.Anotherkindofprojectionistochangethestructureoftheresults:

FORuserINusers

RETURN{userName:user.name,age:user.age}

GettingStarted

Thequerydefinestheoutputformatforeveryuserdocument.TheusernameisreturnedasuserNameinsteadofname,theagekeepstheattributekeyinthisexample:

"userName":"JamesHendrix",

"age":69

"userName":"JohnSmith",

"age":32

"userName":"KatieFoster",

"age":40

Itisalsopossibletocomputenewvalues:

FORuserINusers

RETURNCONCAT(user.name,"'sageis",user.age)

CONCAT()isafunctionthatcanjoinelementstogethertoastring.Weuseitheretoreturnastatementforeveryuser.Asyoucansee,theresultsetdoesnotalwayshavetobeanarrayofobjects:

"JamesHendrix'sageis69",

"JohnSmith'sageis32",

"KatieFoster'sageis40"

Nowlet'sdosomethingcrazy:foreverydocumentintheuserscollection,iterateoveralluserdocumentsagainandreturnuserpairs,e.g.JohnandKatie.Wecanusealoopinsidealoopforthistogetthecrossproduct(everypossiblecombinationofalluserrecords,3*3=9).Wedon'twantpairingslikeJohn+Johnhowever,solet'seliminatethemwithafiltercondition:

FORuser1INusers

FORuser2INusers

FILTERuser1!=user2

RETURN[user1.name,user2.name]

Weget6pairings.PairslikeJames+JohnandJohn+Jamesarebasicallyredundant,butfairenough:

["JamesHendrix","JohnSmith"],

["JamesHendrix","KatieFoster"],

["JohnSmith","JamesHendrix"],

["JohnSmith","KatieFoster"],

["KatieFoster","JamesHendrix"],

["KatieFoster","JohnSmith"]

Wecouldcalculatethesumofbothagesandcomputesomethingnewthisway:

FORuser1INusers

FORuser2INusers

FILTERuser1!=user2

RETURN{

pair:[user1.name,user2.name],

sumOfAges:user1.age+user2.age

WeintroduceanewattributesumOfAgesandaddupbothagesforthevalue:

GettingStarted

"pair":["JamesHendrix","JohnSmith"],

"sumOfAges":101

"pair":["JamesHendrix","KatieFoster"],

"sumOfAges":109

"pair":["JohnSmith","JamesHendrix"],

"sumOfAges":101

"pair":["JohnSmith","KatieFoster"],

"sumOfAges":72

"pair":["KatieFoster","JamesHendrix"],

"sumOfAges":109

"pair":["KatieFoster","JohnSmith"],

"sumOfAges":72

Ifwewantedtopost-filteronthenewattributetoonlyreturnpairswithasumlessthan100,weshoulddefineavariabletotemporarilystorethesum,sothatwecanuseitinaFILTERstatementaswellasintheRETURNstatement:

FORuser1INusers

FORuser2INusers

FILTERuser1!=user2

LETsumOfAges=user1.age+user2.age

FILTERsumOfAges<100

RETURN{

pair:[user1.name,user2.name],

sumOfAges:sumOfAges

TheLETkeywordisfollowedbythedesignatedvariablename(sumOfAges),thenthere'sa=symbolandthevalueoranexpressiontodefinewhatvaluethevariableissupposedtohave.Were-useourexpressiontocalculatethesumhere.WethenhaveanotherFILTERtoskiptheunwantedpairingsandmakeuseofthevariablewedeclaredbefore.Wereturnaprojectionwithanarrayoftheusernamesandthecalculatedage,forwhichweusethevariableagain:

"pair":["JohnSmith","KatieFoster"],

"sumOfAges":72

"pair":["KatieFoster","JohnSmith"],

"sumOfAges":72

Protip:whendefiningobjects,ifthedesiredattributekeyandthevariabletousefortheattributevaluearethesame,youcanuseashorthandnotation:{sumOfAges}insteadof{sumOfAges:sumOfAges}.

Finally,let'sdeleteoneoftheuserdocuments:

REMOVE"9883"INusers

ItdeletestheuserJohn(_key:"9883").Wecouldalsoremovedocumentsinaloop(samegoesforINSERT,UPDATEandREPLACE):

FORuserINusers

FILTERuser.age>=30

REMOVEuserINusers

GettingStarted

Thequerydeletesalluserswhoseageisgreaterthanorequalto30.

Howtocontinue

ThereisalotmoretodiscoverinAQLandmuchmorefunctionalitythatArangoDBoffers.Continuereadingtheotherchaptersandexperimentwithatestdatabasetofosteryourknowledge.

IfyouwanttowritemoreAQLqueriesrightnow,havealookhere:

DataQueries:dataaccessandmodificationqueriesHigh-leveloperations:detaileddescriptionsofFOR,FILTERandmoreoperationsnotshowninthisintroductionFunctions:areferenceofallprovidedfunctions

ArangoDBprogramsTheArangoDBpackagecomeswiththefollowingprograms:

arangod:TheArangoDBdatabasedaemon.ThisserverprogramisintendedtorunasadaemonprocessandtoservethevariousclientsconnectiontotheserverviaTCP/HTTP.

arangosh:TheArangoDBshell.Aclientthatimplementsaread-eval-printloop(REPL)andprovidesfunctionstoaccessandadministratetheArangoDBserver.

arangoimport:AbulkimporterfortheArangoDBserver.ItsupportsJSONandCSV.

arangodump:AtooltocreatebackupsofanArangoDBdatabaseinJSONformat.

arangorestore:AtooltoloaddataofabackupbackintoanArangoDBdatabase.

arango-dfdb:AdatafiledebuggerforArangoDB.ItisprimarilyintendedtobeusedduringdevelopmentofArangoDB.

arangobench:Abenchmarkandtesttool.Itcanbeusedforperformanceandserverfunctiontesting.

GettingStarted

InstallingFirstofall,downloadandinstallthecorrespondingRPMorDebianpackageorusehomebrewonMacOSX.Youcanfindpackagesforvariousoperationsystemsatourinstallsection,includinginstallersforWindows.

Howtodothatindetailisdescribedinthesubchaptersofthissection.

Onhowtosetupacluster,checkouttheDeploymentchapter.

Installing

LinuxVisittheofficialArangoDBinstallpageanddownloadthecorrectpackageforyourLinuxdistribution.Youcanfindbinarypackagesforthemostcommondistributionsthere.Followtheinstructionstouseyourfavoritepackagemanagerforthemajordistributions.AftersettinguptheArangoDBrepositoryyoucaneasilyinstallArangoDBusingyum,aptitude,urpmiorzypper.Debianbasedpackageswillaskforapasswordduringinstallation.ForanunattendedinstallationforDebian,seebelow.Red-Hatbasedpackageswillsetarandompasswordduringinstallation.Forotherdistributionsortochangethepassword,runarango-secure-installationtosetarootpassword.Alternatively,seeCompilingifyouwanttobuildArangoDByourself.Startupthedatabaseserver.

Normally,thisisdonebyexecutingthefollowingcommand:

unix>/etc/init.d/arangodstart

Itwillstarttheserver,anddothataswellatsystemboottime.

Tostoptheserveryoucanusethefollowingcommand:

unix>/etc/init.d/arangodstop

TheexactcommandsdependonyourLinuxdistribution.Youmayrequirerootprivilegestoexecutethesecommands.

LinuxMint

PleaseusethecorrespondingUbuntuorDebianpackages.

UnattendedInstallation

Debianbasedpackagewillaskforapasswordduringinstallation.Forunattendedinstallation,youcansetthepasswordusingthedebconfhelpers.

echoarangodb3arangodb3/passwordpasswordNEWPASSWORD|debconf-set-selections

echoarangodb3arangodb3/password_againpasswordNEWPASSWORD|debconf-set-selections

Thecommandsshouldbeexecutedpriortotheinstallation.

Red-Hatbasedpackageswillsetarandompasswordduringinstallation.Ifyouwanttoforceapassword,execute

ARANGODB_DEFAULT_ROOT_PASSWORD=NEWPASSWORDarango-secure-installation

Thecommandshouldbeexecutedaftertheinstallation.

Non-StandardInstallationIfyoucompiledArangoDBfromsourceanddidnotuseanyinstallationpackage–orusingnon-defaultlocationsand/ormultipleArangoDBinstancesonthesamehost–youmaywanttostarttheserverprocessmanually.Youcandosobyinvokingthearangodbinaryfromthecommandlineasshownbelow:

unix>/usr/local/sbin/arangod/tmp/vocbase

20ZZ-XX-YYT12:37:08Z[8145]INFOusingbuilt-inJavaScriptstartupfiles

20ZZ-XX-YYT12:37:08Z[8145]INFOArangoDB(version1.x.y)isreadyforbusiness

20ZZ-XX-YYT12:37:08Z[8145]INFOHaveFun!

Tostopthedatabaseservergracefully,youcaneitherpressCTRL-CorbysendtheSIGINTsignaltotheserverprocess.Onmanysystemsthiscanbeachievedwiththefollowingcommand:

unix>kill-2`pidofarangod`

Onceyoustartedtheserver,thereshouldbearunninginstanceofarangod-theArangoDBdatabaseserver.

unix>psauxw|fgreparangod

arangodb145360.10.6530726423464s002S1:21pm0:00.18/usr/local/sbin/arangod

Ifthereisnosuchprocess,checkthelogfile/var/log/arangodb/arangod.logforerrors.Ifyouseealogmessagelike

2012-12-03T11:35:29Z[12882]ERRORDatabasedirectoryversion(1)islowerthanserverversion(1.2).

2012-12-03T11:35:29Z[12882]ERRORItseemslikeyouhaveupgradedtheArangoDBbinary.Ifthisiswhatyouwantedtodo,pleas

erestartwiththe--database.auto-upgradeoptiontoupgradethedatainthedatabasedirectory.

2012-12-03T11:35:29Z[12882]FATALDatabaseversioncheckfailed.Pleasestarttheserverwiththe--database.auto-upgradeopti

makesuretostarttheserveroncewiththe--database.auto-upgradeoption.

Notethatyoumayhavetoenableloggingfirst.Ifyoustarttheserverinashell,youshouldseeerrorsloggedthereaswell.

MacOSXThepreferredmethodforinstallingArangoDBunderMacOSXishomebrew.However,incaseyouarenotusinghomebrew,weprovideacommand-lineapporgraphicalappwhichcontainsalltheexecutables.

Homebrew

Ifyouareusinghomebrew,thenyoucaninstallthelatestreleasedstableversionofArangoDBusingbrewasfollows:

brewinstallarangodb

ThiswillinstallthecurrentstableversionofArangoDBandalldependencieswithinyourHomebrewtree.Notethattheserverwillbeinstalledas:

/usr/local/sbin/arangod

Youcanstarttheserverbyrunningthecommand/usr/local/sbin/arangod&.

Configurationfileislocatedat

/usr/local/etc/arangodb3/arangod.conf

TheArangoDBshellwillbeinstalledas:

/usr/local/bin/arangosh

YoucanuninstallArangoDBusing:

brewuninstallarangodb

However,incaseyoustartedArangoDBusingthelaunchctl,youneedtounloaditbeforeuninstallingtheserver:

launchctlunload~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist

ThenremovetheLaunchAgent:

rm~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist

Note:IfthelatestArangoDBVersionisnotshowninhomebrew,youalsoneedtoupdatehomebrew:

brewupdate

Knownissues

Performance-theLLVMdeliveredasofMacOSXElCapitanbuildsslowbinaries.UseGCCinstead,untilthisissuehasbeenfixedbyApple.theCommandlineargumentparsingdoesn'tacceptblanksinfilenames;theCLIversionbelowdoes.ifyouneedtochangeserverendpointwhilestartinghomebrewversion,youcaneditarangod.conffileanduncommentlinewithendpointneeded,e.g.:

[server]

endpoint=tcp://0.0.0.0:8529

MacOSX

GraphicalApp

Incaseyouarenotusinghomebrew,wealsoprovideagraphicalapp.Youcandownloaditfromhere.

ChooseMacOSX.DownloadandinstalltheapplicationArangoDBinyourapplicationfolder.

CommandlineAppIncaseyouarenotusinghomebrew,wealsoprovideacommand-lineapp.Youcandownloaditfromhere.

ChooseMacOSX.DownloadandinstalltheapplicationArangoDB-CLIinyourapplicationfolder.

Startingtheapplicationwillstarttheserverandopenaterminalwindowshowingyouthelog-file.

ArangoDBserverhasbeenstarted

Thedatabasedirectoryislocatedat

'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/lib/arangodb'

Thelogfileislocatedat

'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/log/arangodb/arangod.log'

Youcanaccesstheserverusingabrowserat'http://127.0.0.1:8529/'

orstarttheArangoDBshell

'/Applications/ArangoDB-CLI.app/Contents/MacOS/arangosh'

Switchingtolog-filenow,killingthiswindowswillNOTstoptheserver.

2013-10-27T19:42:04Z[23840]INFOArangoDB(version1.4.devel[darwin])isreadyforbusiness.Havefun!

Notethatitispossibletoinstallboth,thehomebrewversionandthecommand-lineapp.Youshould,however,edittheconfigurationfilesofoneversionandchangetheportused.

MacOSX

WindowsThedefaultinstallationdirectoryisC:\ProgramFiles\ArangoDB-3.x.x.Duringtheinstallationprocessyoumaychangethis.InthefollowingdescriptionwewillassumethatArangoDBhasbeeninstalledinthelocation<ROOTDIR>.

Youhavetobecarefulwhenchoosinganinstallationdirectory.Youneedeitherwritepermissiontothisdirectoryoryouneedtomodifytheconfigurationfilefortheserverprocess.InthelattercasethedatabasedirectoryandtheFoxxdirectoryhavetobewritablebytheuser.

Single-andMultiuserInstallation

TherearetwomainmodesfortheinstallerofArangoDB.Theinstallerletsyouselect:

multiuserinstallation(default;adminprivilegesrequired)WillinstallArangoDBasservice.singleuserinstallationAllowtoinstallArangodbasnormaluser.Requiresmanualstartingofthedatabaseserver.

CheckBoxesThecheckboxesallowyoutochoseweatheryouwantto:

chosecustominstallpathsdoanautomaticupgradekeepanbackupofyourdataaddexecutablestopathcreateadesktopicon

ornot.

CustomInstallPaths

Thischeckboxcontrolsifyouwillbeabletooverridethedefaultpathsfortheinstallationinsubsequentsteps.

Thedefaultinstallationpathsare:

MultiUserDefault:

Installation:C:\ProgramFiles\ArangoDB-3.x.xDataBase:C:\ProgramData\ArangoDBFoxxService:C:\ProgramData\ArangoDB-apps

SingleUserDefault:

Installation:C:\Users\\\AppData\Local\ArangoDB-3.x.xDataBase:C:\Users\\\AppData\Local\ArangoDBFoxxService:C:\Users\\\AppData\Local\ArangoDB-apps

Wearenotusingtheroamingpartoftheuser'sprofile,becausedoingsoavoidsthedatabeingsyncedtothewindowsdomaincontroller.

AutomaticUpgrade

Ifthischeckboxisselectedtheinstallerwillattempttoperformanautomaticupdate.FormoreinformationpleaseseeUpgradingfromPreviousVersion.

KeepBackup

Selectthistocreateabackupofyourdatabasedirectoryduringautomaticupgrade.Thebackupwillbecreatednexttoyourcurrentdatabasedirectorysuffixedbyatimestamp.

Windows

AddtoPath

Selectthistoaddthebinarydirectorytoyoursystem'spath(multiuserinstallation)oruser'spath(singleuserinstallation).

DesktopIcon

SelectifyouwanttheinstallertocreateDesktopIconsthatletyou:

accessthewebintefacestartthecommandlineclient(arangosh)startthedatabaseserver(singleuserinstallationonly)

UpgradingfromPreviousVersionIfyouareupgradingArangoDBfromanearlierversionyouneedtocopyyourolddatabasedirectorytothenewdefaultpaths.Upgradingwillkeepyourolddata,passwordandchoiceofstorageengineasitis.SwitchingtotheRocksDBstorageenginerequiresaexportandreimportofyourdata.

StartingIfyouinstalledArangoDBformultipleusers(asaservice)itisautomaticallystarted.OtherwiseyouneedtousethelinkthatwascreatedonyouDesktopifyouchosetolettheinstallercreatedesktopiconsor

theexecutablearangod.exelocatedin<ROOTDIR>\bin.Thiswillusetheconfigurationfilearangod.conflocatedin<ROOTDIR>\etc\arangodb,whichyoucanadjusttoyourneedsandusethedatadirectory<ROOTDIR>\var\lib\arangodb.Thisistheplacewhereallyourdata(databasesandcollections)willbestoredbydefault.

Pleasechecktheoutputofthearangod.exeexecutablebeforegoingon.Iftheserverstartedsuccessfully,youshouldseealineArangoDBisreadyforbusiness.Havefun!attheendofitsoutput.

Wenowwishtocheckthattheinstallationisworkingcorrectlyandtodothiswewillbeusingtheadministrationwebinterface.Executearangod.exeifyouhavenotalreadydoneso,thenopenupyourwebbrowserandpointittothepage:

http://127.0.0.1:8529/

AdvancedStarting

Ifyouwanttoprovideourownstartscripts,youcansettheenvironmentvariableARANGODB_CONFIG_PATH.Thisvariableshouldpointtoadirectorycontainingtheconfigurationfiles.

UsingtheClient

ToconnecttoanalreadyrunningArangoDBserverinstance,thereisashellarangosh.exelocatedin<ROOTDIR>\bin.Thisstartsashellwhichcanbeused–amongstotherthings–toadministerandqueryalocalorremoteArangoDBserver.

Notethatarangosh.exedoesNOTstartaseparateserver,itonlystartstheshell.Touseityoumusthaveaserverrunningsomewhere,e.g.byusingthearangod.exeexecutable.

arangosh.exeusesconfigurationfromthefilearangosh.conflocatedin<ROOTDIR>\etc\arangodb\.Pleaseadjustthistoyourneedsifyouwanttousedifferentconnectionsettingsetc.

Uninstalling

TouninstalltheArangoserverapplicationyoucanusethewindowscontrolpanel(asyouwouldnormallyuninstallanapplication).Notehowever,thatanydatafilescreatedbytheArangoserverwillremainaswellasthe<ROOTDIR>directory.Tocompletetheuninstallationprocess,removethedatafilesandthe<ROOTDIR>directorymanually.

Windows

LimitationsforCygwin

PleasenotesomeimportantlimitationswhenrunningArangoDBunderCygwin:StartingArangoDBcanbestartedfromoutofaCygwinterminal,butpressingCTRL-Cwillforcefullykilltheserverprocesswithoutgivingitachancetohandlethekillsignal.Inthiscase,aregularservershutdownisnotpossible,whichmayleaveafileLOCKaroundintheserver'sdatadirectory.ThisfileneedstoberemovedmanuallytomakeArangoDBstartagain.Additionally,asArangoDBdoesnothaveachancetohandlethekillsignal,theservercannotforcefullyflushanydatatodiskonshutdown,leadingtopotentialdataloss.WhenstartingArangoDBfromaCygwinterminalitmightalsohappenthatnoerrorsareprintedintheterminaloutput.StartingArangoDBfromanMS-DOScommandpromptdoesnotimposetheselimitationsandisthusthepreferredmethod.

PleasenotethatArangoDBusesUTF-8asitsinternalencodingandthatthesystemconsolemustsupportaUTF-8codepage(65001)andfont.ItmaybenecessarytomanuallyswitchtheconsolefonttoafontthatsupportsUTF-8.

Windows

CompilingArangoDBfromscratchThefollowingsectionsdescribehowtocompileandbuildtheArangoDBfromscratch.ArangoDBwillcompileonmostLinuxandMacOSXsystems.WeassumethatyouusetheGNUC/C++compilerorclang/clang++tocompilethesource.ArangoDBhasbeentestedwiththesecompilers,butshouldbeabletocompilewithanyPosix-compliant,C++11-enabledcompiler.PleaseletusknowwhetheryousuccessfullycompileditwithanotherC/C++compiler.

Bydefault,cloningthegithubrepositorywillcheckoutdevel.ThisversioncontainsthedevelopmentversionoftheArangoDB.UsethisbranchifyouwanttomakechangestotheArangoDBsource.

OnWindowsyoufirstneedtoallowandenablesymlinksforyouruser.

PleasecheckoutthecookbookonhowtocompileArangoDB.

Compiling

AuthenticationArangoDBallowstorestrictaccesstodatabasestocertainusers.Allusersofthesystemdatabaseareconsideredadministrators.Duringinstallationadefaultuserrootiscreated,whichhasaccesstoalldatabases.

Youshouldcreateadatabaseforyourapplicationtogetherwithauserthathasaccessrightstothisdatabase.SeeManagingUsers.

Usethearangoshtocreateanewdatabaseanduser.

arangosh>db._createDatabase("example");

arangosh>varusers=require("@arangodb/users");

arangosh>users.save("root@example","password");

arangosh>users.grantDatabase("root@example","example");

Youcannowconnecttothenewdatabaseusingtheuserroot@example.

shell>arangosh--server.username"root@example"--server.databaseexample

Authentication

AccessingtheWebInterfaceArangoDBcomeswithabuilt-inwebinterfaceforadministration.ThewebinterfacecanbeaccessedviatheURL:

http://127.0.0.1:8529

Ifeverythingworksasexpected,youshouldseetheloginview:

FormoreinformationontheArangoDBwebinterface,seeWebInterface

AccessingtheWebInterface

ComingfromSQLIfyouworkedwitharelationaldatabasemanagementsystem(RDBMS)suchasMySQL,MariaDBorPostgreSQL,youwillbefamiliarwithitsquerylanguage,adialectofSQL(StructuredQueryLanguage).

ArangoDB'squerylanguageiscalledAQL.Therearesomesimilaritiesbetweenbothlanguagesdespitethedifferentdatamodelsofthedatabasesystems.ThemostnotabledifferenceisprobablytheconceptofloopsinAQL,whichmakesitfeelmorelikeaprogramminglanguage.Itsuitestheschema-lessmodelmorenaturalandmakesthequerylanguageverypowerfulwhileremainingeasytoreadandwrite.

TogetstartedwithAQL,havealookatourdetailedcomparisonofSQLandAQL.ItwillalsohelpyoutotranslateSQLqueriestoAQLwhenmigratingtoArangoDB.

Howdobrowsevectorstranslateintodocumentqueries?

IntraditionalSQLyoumayeitherfetchallcolumnsofatablerowbyrow,usingSELECT*FROMtable,orselectasubsetofthecolumns.Thelistoftablecolumnstofetchiscommonlycalledcolumnlistorbrowsevector:

SELECTcolumnA,columnB,columnZFROMtable

Sincedocumentsaren'ttwo-dimensional,andneitherdoyouwanttobelimitedtoreturningtwo-dimensionallists,therequirementsforaquerylanguagearehigher.AQListhusalittlebitmorecomplexthanplainSQLatfirst,butoffersmuchmoreflexibilityinthelongrun.Itletsyouhandlearbitrarilystructureddocumentsinconvenientways,mostlyleanedonthesyntaxusedinJavaScript.

Composingthedocumentstobereturned

TheAQLRETURNstatementreturnsoneitemperdocumentitishanded.Youcanreturnthewholedocument,orjustpartsofit.GiventhatoneDocumentisadocument(retrievedlikeLEToneDocument=DOCUMENT("myusers/3456789")forinstance),itcanbereturnedas-islikethis:

RETURNoneDocument

"_id":"myusers/3456789",

"_key":"3456789"

"_rev":"14253647",

"firstName":"John",

"lastName":"Doe",

"address":{

"city":"Gotham",

"street":"RoadToNowhere1"

"hobbies":[

{name:"swimming",howFavorite:10},

{name:"biking",howFavorite:6},

{name:"programming",howFavorite:4}

Returnthehobbiessub-structureonly:

RETURNoneDocument.hobbies

ComingfromSQL

Returnthehobbiesandtheaddress:

RETURN{

hobbies:oneDocument.hobbies,

address:oneDocument.address

hobbies:[

address:{

"city":"Gotham",

"street":"RoadToNowhere1"

Returnthefirsthobbyonly:

RETURNoneDocument.hobbies[0].name

"swimming"

Returnalistofallhobbystrings:

RETURN{hobbies:oneDocument.hobbies[*].name}

{hobbies:["swimming","biking","porgramming"]}

MorecomplexarrayandobjectmanipulationscanbedoneusingAQLfunctionsandoperators.

ComingfromSQL

Highlights

Version3.3

EnterpriseEdition

DatacentertoDatacenterReplication:ReplicatetheentirestructureandcontentofanArangoDBclusterasynchronouslytoanotherclusterinadifferentdatacenterwithArangoSync.Multi-datacentersupportmeansyoucanfallbacktoareplicaofyourclusterincaseofadisasterinonedatacenter.

EncryptedBackups:ArangodumpcancreatebackupsencryptedwithasecretkeyusingAES256blockcipher.

AllEditions

Server-levelReplication:Inadditiontoper-databasereplication,thereisnowanadditionalglobalApplier.Starttheglobalreplicationontheslaveonceandallcurrentandfuturedatabaseswillbereplicatedfromthemastertotheslaveautomatically.

AsynchronousFailover:Makeasingleserverinstanceresilientwithasecondserverinstance,oneasmasterandtheotherasasynchronouslyreplicatingslave,withautomaticfailovertotheslaveifthemastergoesdown.

AlsoseeWhat'sNewin3.3.

Version3.2

RocksDBStorageEngine:YoucannowuseasmuchdatainArangoDBasyoucanfitonyourdisk.Plus,youcanenjoyperformanceboostsonwritesbyhavingonlydocument-levellocks

Pregel:WeimplementeddistributedgraphprocessingwithPregeltodiscoverhiddenpatterns,identifycommunitiesandperformin-depthanalyticsoflargegraphdatasets.

Fault-TolerantFoxx:TheFoxxmanagementinternalshavebeenrewrittenfromthegrounduptomakesuremulti-coordinatorclustersetupsalwayskeeptheirservicesinsyncandnewcoordinatorsarefullyinitializedevenwhenallexistingcoordinatorsareunavailable.

Enterprise:Workingwithsomeofourlargestcustomers,we’veaddedfurthersecurityandscalabilityfeaturestoArangoDBEnterpriselikeLDAPintegration,EncryptionatRest,andthebrandnewSatelliteCollections.

Version3.1SmartGraphs:Scalewithgraphstoaclusterandstayperformant.WithSmartGraphsyoucanusethe"smartness"ofyourapplicationlayertoshardyourgraphefficientlytoyourmachinesandlettraversalsrunlocally.

EncryptionControl:ChooseyourlevelofSSLencryption

Auditing:KeepadetailedlogofalltheimportantthingsthathappenedinArangoDB.

Version3.0

self-organizingclusterwithsynchronousreplication,master/mastersetup,sharednothingarchitecture,clustermanagementagency.

Deeplyintegrated,nativeAQLgraphtraversal

Highlights

VelocyPackasnewinternalbinarystorageformataswellasforintermediateAQLvalues.

PersistentindexesviaRocksDBsuitableforsortingandrangequeries.

Foxx3.0:overhauledJSframeworkfordata-centricmicroservices

SignificantlyimprovedWebInterface

Highlights

ScalabilityArangoDBisadistributeddatabasesupportingmultipledatamodels,andcanthusbescaledhorizontally,thatis,byusingmanyservers,typicallybasedoncommodityhardware.Thisapproachnotonlydeliversperformanceaswellascapacityimprovements,butalsoachievesresiliencebymeansofreplicationandautomaticfail-over.Furthermore,onecanbuildsystemsthatscaletheircapacitydynamicallyupanddownautomaticallyaccordingtodemand.

OnecanalsoscaleArangoDBvertically,thatis,byusingeverlargerservers.ThereisnobuiltinlimitationinArangoDB,forexample,theserverwillautomaticallyusemorethreadsifmoreCPUsarepresent.

However,scalingverticallyhasthedisadvantagethatthecostsgrowfasterthanlinearwiththesizeoftheserver,andnoneoftheresilienceanddynamicalcapabilitiescanbeachievedinthisway.

InthischapterweexplainthedistributedarchitectureofArangoDBanddiscussitsscalabilityfeaturesandlimitations:

ArangoDB'sdistributedarchitectureDifferentdatamodelsandscalabilityLimitations

Scalability

Architecture

TheclusterarchitectureofArangoDBisaCPmaster/mastermodelwithnosinglepointoffailure.With"CP"wemeanthatinthepresenceofanetworkpartition,thedatabaseprefersinternalconsistencyoveravailability.With"master/master"wemeanthatclientscansendtheirrequeststoanarbitrarynode,andexperiencethesameviewonthedatabaseregardless."Nosinglepointoffailure"meansthattheclustercancontinuetoserverequests,evenifonemachinefailscompletely.

Inthisway,ArangoDBhasbeendesignedasadistributedmulti-modeldatabase.Thissectiongivesashortoutlineontheclusterarchitectureandhowtheabovefeaturesandcapabilitiesareachieved.

StructureofanArangoDBcluster

AnArangoDBclusterconsistsofanumberofArangoDBinstanceswhichtalktoeachotheroverthenetwork.Theyplaydifferentroles,whichwillbeexplainedindetailbelow.Thecurrentconfigurationoftheclusterisheldinthe"Agency",whichisahighly-availableresilientkey/valuestorebasedonanoddnumberofArangoDBinstancesrunningRaftConsensusProtocol.

ForthevariousinstancesinanArangoDBclusterthereare4distinctroles:Agents,Coordinators,PrimaryandSecondaryDBservers.Inthefollowingsectionswewillshedlightoneachofthem.NotethatthetasksforallrolesrunthesamebinaryfromthesameDockerimage.

Agents

OneormultipleAgentsformtheAgencyinanArangoDBcluster.TheAgencyisthecentralplacetostoretheconfigurationinacluster.Itperformsleaderelectionsandprovidesothersynchronizationservicesforthewholecluster.WithouttheAgencynoneoftheothercomponentscanoperate.

Whilegenerallyinvisibletotheoutsideitistheheartofthecluster.Assuch,faulttoleranceisofcourseamusthavefortheAgency.ToachievethattheAgentsareusingtheRaftConsensusAlgorithm.ThealgorithmformallyguaranteesconflictfreeconfigurationmanagementwithintheArangoDBcluster.

AtitscoretheAgencymanagesabigconfigurationtree.Itsupportstransactionalreadandwriteoperationsonthistree,andotherserverscansubscribetoHTTPcallbacksforallchangestothetree.

Coordinators

Coordinatorsshouldbeaccessiblefromtheoutside.Thesearetheonestheclientstalkto.TheywillcoordinateclustertaskslikeexecutingqueriesandrunningFoxxservices.Theyknowwherethedataisstoredandwilloptimizewheretorunusersuppliedqueriesorpartsthereof.Coordinatorsarestatelessandcanthuseasilybeshutdownandrestartedasneeded.

PrimaryDBservers

PrimaryDBserversaretheoneswherethedataisactuallyhosted.Theyhostshardsofdataandusingsynchronousreplicationaprimarymayeitherbeleaderorfollowerforashard.

Theyshouldnotbeaccessedfromtheoutsidebutindirectlythroughthecoordinators.Theymayalsoexecutequeriesinpartorasawholewhenaskedbyacoordinator.

Secondaries

SecondaryDBserversareasynchronousreplicasofprimaries.Ifoneisusingonlysynchronousreplication,onedoesnotneedsecondariesatall.Foreachprimary,therecanbeoneormoresecondaries.Sincethereplicationworksasynchronously(eventualconsistency),thereplicationdoesnotimpedetheperformanceoftheprimaries.Ontheotherhand,theirreplicaofthedatacanbeslightlyoutofdate.Thesecondariesareperfectlysuitableforbackupsastheydon'tinterferewiththenormalclusteroperation.

ClusterID

Everynon-AgencyArangoDBinstanceinaclusterisassignedauniqueIDduringitsstartup.UsingitsIDanodeisidentifiablethroughoutthecluster.AllclusteroperationswillcommunicateviathisID.

Architecture

Sharding

UsingtherolesoutlinedaboveanArangoDBclusterisabletodistributedatainsocalledshardsacrossmultipleprimaries.Fromtheoutsidethisprocessisfullytransparentandassuchweachievethegoalsofwhatothersystemscall"master-masterreplication".InanArangoDBclusteryoutalktoanycoordinatorandwheneveryoureadorwritedataitwillautomaticallyfigureoutwherethedataisstored(read)ortobestored(write).TheinformationabouttheshardsissharedacrossthecoordinatorsusingtheAgency.

AlsoseeShardingintheAdministrationchapter.

Manysensibleconfigurations

Thisarchitectureisveryflexibleandthusallowsmanyconfigurations,whicharesuitablefordifferentusagescenarios:

1. ThedefaultconfigurationistorunexactlyonecoordinatorandoneprimaryDBserveroneachmachine.Thisachievestheclassicalmaster/mastersetup,sincethereisaperfectsymmetrybetweenthedifferentnodes,clientscanequallywelltalktoanyoneofthecoordinatorsandallexposethesameviewtothedatastore.

2. OnecandeploymorecoordinatorsthanDBservers.ThisisasensibleapproachifoneneedsalotofCPUpowerfortheFoxxservices,becausetheyrunonthecoordinators.

3. OnecandeploymoreDBserversthancoordinatorsifmoredatacapacityisneededandthequeryperformanceisthelesserbottleneck

4. Onecandeployacoordinatoroneachmachinewhereanapplicationserver(e.g.anode.jsserver)runs,andtheAgentsandDBserversonaseparatesetofmachineselsewhere.Thisavoidsanetworkhopbetweentheapplicationserverandthedatabaseandthusdecreaseslatency.Essentially,thismovessomeofthedatabasedistributionlogictothemachinewheretheclientruns.

Theseforshallsufficefornow.TheimportantpieceofinformationhereisthatthecoordinatorlayercanbescaledanddeployedindependentlyfromtheDBserverlayer.

Replication

ArangoDBofferstwowaysofdatareplicationwithinacluster,synchronousandasynchronous.Inthissectionweexplainsomedetailsandhighlighttheadvantagesanddisadvantagesrespectively.

Synchronousreplicationwithautomaticfail-over

Synchronousreplicationworksonaper-shardbasis.Oneconfiguresforeachcollection,howmanycopiesofeachshardarekeptinthecluster.Atanygiventime,oneofthecopiesisdeclaredtobethe"leader"andallotherreplicasare"followers".WriteoperationsforthisshardarealwayssenttotheDBserverwhichhappenstoholdtheleadercopy,whichinturnreplicatesthechangestoallfollowersbeforetheoperationisconsideredtobedoneandreportedbacktothecoordinator.Readoperationsareallservedbytheserverholdingtheleadercopy,thisallowstoprovidesnapshotsemanticsforcomplextransactions.

IfaDBserverfailsthatholdsafollowercopyofashard,thentheleadercannolongersynchronizeitschangestothatfollower.Afterashorttimeout(3seconds),theleadergivesuponthefollower,declaresittobeoutofsync,andcontinuesservicewithoutthefollower.Whentheserverwiththefollowercopycomesback,itautomaticallyresynchronizesitsdatawiththeleaderandsynchronousreplicationisrestored.

IfaDBserverfailsthatholdsaleadercopyofashard,thentheleadercannolongerserveanyrequests.ItwillnolongersendaheartbeattotheAgency.Therefore,asupervisionprocessrunningintheRaftleaderoftheAgency,cantakethenecessaryaction(after15secondsofmissingheartbeats),namelytopromoteoneoftheserversthatholdin-syncreplicasoftheshardtoleaderforthatshard.ThisinvolvesareconfigurationintheAgencyandleadstothefactthatcoordinatorsnowcontactadifferentDBserverforrequeststothisshard.Serviceresumes.Theothersurvivingreplicasautomaticallyresynchronizetheirdatawiththenewleader.WhentheDBserverwiththeoriginalleadercopycomesback,itnoticesthatitnowholdsafollowerreplica,resynchronizesitsdatawiththenewleaderandorderisrestored.

Allsharddatasynchronizationsaredoneinanincrementalway,suchthatresynchronizationsarequick.Thistechnologyallowstomoveshards(followerandleaderones)betweenDBserverswithoutserviceinterruptions.Therefore,anArangoDBclustercanmoveallthedataonaspecificDBservertootherDBserversandthenshutdownthatserverinacontrolledway.ThisallowstoscaledownanArangoDBclusterwithoutserviceinterruption,lossoffaulttoleranceordataloss.Furthermore,onecanre-balancethedistributionoftheshards,eithermanuallyorautomatically.

AlltheseoperationscanbetriggeredviaaREST/JSONAPIorviathegraphicalwebUI.Allfail-overoperationsarecompletelyhandledwithintheArangoDBcluster.

Architecture

Obviously,synchronousreplicationinvolvesacertainincreasedlatencyforwriteoperations,simplybecausethereisonemorenetworkhopwithintheclusterforeveryrequest.Thereforetheusercansetthereplicationfactorto1,whichmeansthatonlyonecopyofeachshardiskept,therebyswitchingoffsynchronousreplication.Thisisasuitablesettingforlessimportantoreasilyrecoverabledataforwhichlowlatencywriteoperationsmatter.

Asynchronousreplicationwithautomaticfail-over

Asynchronousreplicationworksdifferently,inthatitisorganizedusingprimaryandsecondaryDBservers.Eachsecondaryserverreplicatesallthedataheldonaprimarybypollinginanasynchronousway.Thisprocesshasverylittleimpactontheperformanceoftheprimary.Thedisadvantageisthatthereisadelaybetweentheconfirmationofawriteoperationthatissenttotheclientandtheactualreplicationofthedata.Ifthemasterserverfailsduringthisdelay,thencommittedandconfirmeddatacanbelost.

Nevertheless,wealsoofferautomaticfail-overwiththissetup.Contrarytothesynchronouscase,herethefail-overmanagementisdonefromoutsidetheArangoDBcluster.InafutureversionwemightmovethismanagementintothesupervisionprocessintheAgency,butasofnow,themanagementisdoneviatheMesosframeworkschedulerforArangoDB(seebelow).

ThegranularityofthereplicationisawholeArangoDBinstancewithalldatathatresidesonthatinstance,whichmeansthatyouneedtwiceasmanyinstancesaswithoutasynchronousreplication.Synchronousreplicationismoreflexibleinthatrespect,youcanhavesmallerandlargerinstances,andifonefails,thedatacanberebalancedacrosstheremainingones.

Microservicesandzeroadministation

ThedesignandcapabilitiesofArangoDBaregearedtowardsusageinmodernmicroservicearchitecturesofapplications.WiththeFoxxservicesitisveryeasytodeployadatacentricmicroservicewithinanArangoDBcluster.

Inaddition,onecandeploymultipleinstancesofArangoDBwithinthesameproject.Onepartoftheprojectmightneedascalabledocumentstore,anothermightneedagraphdatabase,andyetanothermightneedthefullpowerofamulti-modeldatabaseactuallymixingthevariousdatamodels.Thereareenormousefficiencybenefitstobereapedbybeingabletouseasingletechnologyforvariousrolesinaproject.

TosimplifyliveofthedevopsinsuchascenariowetryasmuchaspossibletouseazeroadministrationapproachforArangoDB.ArunningArangoDBclusterisresilientagainstfailuresandessentiallyrepairsitselfincaseoftemporaryfailures.Seethenextsectionforfurthercapabilitiesinthisdirection.

ApacheMesosintegration

Forthedistributedsetup,weusetheApacheMesosinfrastructurebydefault.ArangoDBisafullycertifiedpackageforDC/OSandcanthusbedeployedessentiallywithafewmouseclicksorasinglecommand,onceyouhaveanexistingDC/OScluster.ButevenonaplainApacheMesosclusteronecandeployArangoDBviaMarathonwithasingleAPIcallandsomeJSONconfiguration.

Theadvantageofthisapproachisthatwecannotonlyimplementtheinitialdeployment,butalsothelatermanagementofautomaticreplacementoffailedinstancesandthescalingoftheArangoDBcluster(triggeredmanuallyorevenautomatically).SinceallmanipulationsareeitherviathegraphicalwebUIorviaJSON/RESTcalls,onecanevenimplementauto-scalingveryeasily.

ADC/OSclusterisaverynaturalenvironmenttodeploymicroservicearchitectures,sinceitissoconvenienttodeployvariousservices,includingpotentiallymultipleArangoDBclusterinstanceswithinthesameDC/OScluster.Thebuilt-inservicediscoverymakesitextremelysimpletoconnectthevariousmicroservicesandMesosautomaticallytakescareofthedistributionanddeploymentofthevarioustasks.

SeetheDeploymentchapteranditssubsectionsforinstructions.

ItispossibletodeployanArangoDBclusterbysimplylaunchingabunchofDockercontainerswiththerightcommandlineoptionstolinkthemup,orevenonasinglemachinestartingmultipleArangoDBprocesses.Inthatcase,synchronousreplicationwillworkwithinthedeployedArangoDBcluster,andautomaticfail-overinthesensethatthedutiesofafailedserverwillautomaticallybeassignedtoanother,survivingone.However,sincetheArangoDBclustercannotwithinitselflaunchadditionalinstances,replacementoffailednodesisnotautomaticandscalingupanddownhastobemanagedmanually.Thisiswhywedonotrecommendthissetupforproductiondeployment.

Architecture

Differentdatamodelsandscalability

InthissectionwediscussscalabilityinthecontextofthedifferentdatamodelssupportedbyArangoDB.

Key/valuepairs

Thekey/valuestoredatamodelistheeasiesttoscale.InArangoDB,thisisimplementedinthesensethatadocumentcollectionalwayshasaprimarykey_keyattributeandintheabsenceoffurthersecondaryindexesthedocumentcollectionbehaveslikeasimplekey/valuestore.

Theonlyoperationsthatarepossibleinthiscontextaresinglekeylookupsandkey/valuepairinsertionsandupdates.If_keyistheonlyshardingattributethentheshardingisdonewithrespecttotheprimarykeyandalltheseoperationsscalelinearly.Iftheshardingisdoneusingdifferentshardkeys,thenalookupofasinglekeyinvolvesaskingallshardsandthusdoesnotscalelinearly.

Documentstore

Forthedocumentstorecaseeveninthepresenceofsecondaryindexesessentiallythesameargumentsapply,sinceanindexforashardedcollectionissimplythesameasalocalindexforeachshard.Therefore,singledocumentoperationsstillscalelinearlywiththesizeofthecluster,unlessaspecialshardingconfigurationmakeslookupsorwriteoperationsmoreexpensive.

ForadeeperanalysisofthistopicseethisblogpostinwhichgoodlinearscalabilityofArangoDBforsingledocumentoperationsisdemonstrated.

Complexqueriesandjoins

TheAQLquerylanguageallowscomplexqueries,usingmultiplecollections,secondaryindexesaswellasjoins.Inparticularwiththelatter,scalingcanbeachallenge,sinceifthedatatobejoinedresidesondifferentmachines,alotofcommunicationhastohappen.TheAQLqueryexecutionengineorganizesadatapipelineacrosstheclustertoputtogethertheresultsinthemostefficientway.Thequeryoptimizerisawareoftheclusterstructureandknowswhatdataiswhereandhowitisindexed.Therefore,itcanarriveataninformeddecisionaboutwhatpartsofthequeryoughttorunwhereinthecluster.

Nevertheless,forcertaincomplicatedjoins,therearelimitsastowhatcanbeachieved.

Graphdatabase

Graphdatabasesareparticularlygoodatqueriesongraphsthatinvolvepathsinthegraphofanaprioriunknownlength.Forexample,findingtheshortestpathbetweentwoverticesinagraph,orfindingallpathsthatmatchacertainpatternstartingatagivenvertexaresuchexamples.

However,iftheverticesandedgesalongtheoccurringpathsaredistributedacrossthecluster,thenalotofcommunicationisnecessarybetweennodes,andperformancesuffers.Toachievegoodperformanceatscale,itisthereforenecessarytogetthedistributionofthegraphdataacrosstheshardsintheclusterright.Mostofthetime,theapplicationdevelopersandusersofArangoDBknowbest,howtheirgraphsarastructured.Therefore,ArangoDBallowsuserstospecify,accordingtowhichattributesthegraphdataissharded.Ausefulfirststepisusuallytomakesurethattheedgesoriginatingatavertexresideonthesameclusternodeasthevertex.

Datamodels

Limitations

ArangoDBhasnobuilt-inlimitationstohorizontalscalability.ThecentralresilientAgencywilleasilysustainhundredsofDBserversandcoordinators,andtheusualdatabaseoperationsworkcompletelydecentrallyanddonotrequireassistanceoftheAgency.

Likewise,thesupervisionprocessintheAgencycaneasilydealwithlotsofservers,sinceallitsactivitiesarenotperformancecritical.

Obviously,anArangoDBclusterislimitedbytheavailableresourcesofCPU,memory,diskandnetworkbandwidthandlatency.

Limitations

DatacentertodatacenterreplicationThischapterintroducesArangoDB'sdatacentertodatacenterreplication(DC2DC).

Forfurtherinformationaboutdatacentertodatacenterreplication,pleaserefertothefollowingsections:

DeploymentAdministrationTroubleshootingMonitoringSecurity

IntroductionAtsomepointinthegrowsofadatabase,therecomesaneedforreplicatingitacrossmultipledatacenters.

Reasonsforthatcanbe:

FallbackincaseofadisasterinonedatacenterRegionalavailabilitySeparationofconcerns

Andmanymore.

Startingfromversion3.3,ArangoDBsupportsdatacentertodatacenterreplication,viatheArangoSynctool.

ArangoDB'sdatacentertodatacenterreplicationisasolutionthatenablesyoutoasynchronouslyreplicatetheentirestructureandcontentinanArangoDBClusterinoneplacetoaClusterinanotherplace.Typicallyitisusedfromonedatacentertoanother.Itisnotasolutionforreplicatingsingleserverinstances.

ThereplicationdonebyArangoSyncinasynchronous.Thatmeansthatwhenaclientiswritingdataintothesourcedatacenter,itwillconsidertherequestfinishedbeforethedatahasbeenreplicatedtotheotherdatacenter.Thetimeneededtocompletelyreplicatechangestotheotherdatacenteristypicallyintheorderofseconds,butthiscanvarysignificantlydependingonload,network&computercapacity.

ArangoSyncperformsreplicationinasingledirectiononly.ThatmeansthatyoucanreplicatedatafromclusterAtoclusterBorfromclusterBtoclusterA,butneveratthesametime.Datamodifiedinthedestinationclusterwillbelost!

Replicationisacompletelyautonomousprocess.Onceitisconfigureditisdesignedtorun24/7withoutfrequentmanualintervention.Thisdoesnotmeanthatitrequiresnomaintenanceorattentionatall.Aswithanydistributedsystemsomeattentionisneededtomonitoritsoperationandkeepitsecure(e.g.certificate&passwordrotation).

Onceconfigured,ArangoSyncwillreplicatebothstructureanddataofanentirecluster.Thismeansthatthereisnoneedtomakeadditionalconfigurationchangeswhenadding/removingdatabasesorcollections.Alsometadatasuchasusers,foxxapplication&jobsareautomaticallyreplicated.

Introduction

Whentouseit...andwhennotThedatacentertodatacenterreplicationisagoodsolutioninallcaseswhereyouwanttoreplicatedatafromoneclustertoanotherwithouttherequirementthatthedataisavailableimmediatelyintheothercluster.

Thedatacentertodatacenterreplicationisnotagoodsolutionwhenoneofthefollowingapplies:

YouwanttoreplicatedatafromclusterAtoclusterBandfromclusterBtoclusterAatthesametime.Youneedsynchronousreplicationbetween2clusters.ThereisnonetworkconnectionbetweenclusterAandB.Youwantcompletecontroloverwhichdatabase,collection&documentsarereplicateandwhichnot.

Applicability

RequirementsTousedatacentertodatacenterreplicationyouneedthefollowing:

Twodatacenters,eachrunninganArangoDBEnterprisecluster,version3.3orhigher.Anetworkconnectionbetweenbothdatacenterswithaccessibleendpointsforseveralcomponents(seeindividualcomponentsfordetails).TLScertificatesforArangoSyncmasterinstances(canbeself-signed).TLScertificatesforKafkabrokers(canbeself-signed).Optional(butrecommended)TLScertificatesforArangoDBclusters(canbeself-signed).ClientcertificatesCAforArangoSyncmasters(typicallyself-signed).ClientcertificatesforArangoSyncmasters(typicallyself-signed).Atleast2instancesoftheArangoSyncmasterineachdatacenter.OneinstancesoftheArangoSyncworkeroneverymachineineachdatacenter.

Note:Inseveralplacesyouwillneeda(x509)certificate.TheCertificatessectionprovidesmoreguidanceforcreatingandrenewingthesecertificates.

Besidestheabovelist,youprobablywanttousethefollowing:

Anorchestratortokeepallcomponentsrunning,e.g.systemd.Alogfilecollectorforcentralizedcollection&accesstothelogsofallcomponents.Ametricscollector&viewingsolutionsuchasPrometheus+Grafana.

Requirements

Datamodels&modelingThischapterintroducesArangoDB'scoreconceptsandcovers

itsdatamodel(ordatamodelsrespectively),theterminologyusedthroughoutthedatabasesystemandinthisdocumentation,aswellasaspectstoconsiderwhenmodelingyourdatatostrikeabalancebetweennaturaldatastructuresandgreatperformance

Youwillalsofindusageexamplesonhowtointeractwiththedatabasesystemusingarangosh,e.g.howtocreateanddropdatabases/collections,orhowtosave,update,replaceandremovedocuments.Youcandoallthisusingthewebinterfaceaswellandmaythereforeskipthesesectionsasbeginner.

Datamodels&modeling

Concepts

DatabaseInteraction

ArangoDBisadatabasethatservesdocumentstoclients.ThesedocumentsaretransportedusingJSONviaaTCPconnection,usingtheHTTPprotocol.ARESTAPIisprovidedtointeractwiththedatabasesystem.

ThewebinterfacethatcomeswithArangoDB,calledAardvark,providesgraphicaluserinterfacethatiseasytouse.Aninteractiveshell,calledArangosh,isalsoshipped.Inaddition,therearesocalleddriversthatmakeiteasytousethedatabasesysteminvariousenvironmentsandprogramminglanguages.AllthesetoolsusetheHTTPinterfaceoftheserverandremovethenecessitytorollownlow-levelcodeforbasiccommunicationinmostcases.

DatamodelThedocumentsyoucanstoreinArangoDBcloselyfollowtheJSONformat,althoughtheyarestoredinabinaryformatcalledVelocyPack.Adocumentcontainszeroormoreattributes,eachoftheseattributeshavingavalue.Avaluecaneitherbeanatomictype,i.e.number,string,booleanornull,oracompoundtype,i.e.anarrayorembeddeddocument/object.Arraysandsub-objectscancontainallofthesetypes,whichmeansthatarbitrarilynesteddatastructurescanberepresentedinasingledocument.

Documentsaregroupedintocollections.Acollectioncontainszeroormoredocuments.Ifyouarefamiliarwithrelationaldatabasemanagementsystems(RDBMS)thenitissafetocomparecollectionstotablesanddocumentstorows.ThedifferenceisthatinatraditionalRDBMS,youhavetodefinecolumnsbeforeyoucanstorerecordsinatable.Suchdefinitionsarealsoknownasschemas.ArangoDBisschema-less,whichmeansthatthereisnoneedtodefinewhatattributesadocumentcanhave.Everysingledocumentcanhaveacompletelydifferentstructureandstillbestoredtogetherwithotherdocumentsinasinglecollection.Inpractice,therewillbecommondenominatorsamongthedocumentsinacollection,butthedatabasesystemitselfdoesn'tforceyoutolimityourselftoacertaindatastructure.

Therearetwotypesofcollections:documentcollection(alsoreferedtoasvertexcollectionsinthecontextofgraphs)aswellasedgecollections.Edgecollectionsstoredocumentsaswell,buttheyincludetwospecialattributes,_fromand_to,whichareusedtocreaterelationsbetweendocuments.Usually,twodocuments(vertices)storedindocumentcollectionsarelinkedbyadocument(edge)storedinanedgecollection.ThisisArangoDB'sgraphdatamodel.Itfollowsthemathematicalconceptofadirected,labeledgraph,exceptthatedgesdon'tjusthavelabels,butarefull-blowndocuments.

Collectionsexistinsideofdatabases.Therecanbeoneormanydatabases.Differentdatabasesareusuallyusedformultitenantsetups,asthedatainsidethem(collections,documentsetc.)isisolatedfromoneanother.Thedefaultdatabase_systemisspecial,becauseitcannotberemoved.Databaseusersaremanagedinthisdatabase,andtheircredentialsarevalidforalldatabasesofaserverinstance.

Similarlydatabasesmayalsocontainviewentities.AViewinitssimplestformcanbeseenasaread-onlyarrayorcollectionofdocuments.Theviewconceptquitecloselymatchesasimilarlynamedconceptavailableinmostrelationaldatabasemanagementsystems(RDBMS).Eachviewentityusuallymapssomeimplementationspecificdocumenttransformation,(possiblyidentity),ontodocumentsfromzeroormorecollections.

DataRetrieval

Queriesareusedtofilterdocumentsbasedoncertaincriteria,tocomputenewdata,aswellastomanipulateordeleteexistingdocuments.Queriescanbeassimpleasa"querybyexample"orascomplexas"joins"usingmanycollectionsortraversinggraphstructures.TheyarewrittenintheArangoDBQueryLanguage(AQL).

Cursorsareusedtoiterateovertheresultofqueries,sothatyougeteasilyprocessablebatchesinsteadofonebighunk.

Indexesareusedtospeedupsearches.Therearevarioustypesofindexes,suchashashindexesandgeoindexes.

Concepts

HandlingDatabasesThisisanintroductiontomanagingdatabasesinArangoDBfromwithinJavaScript.

WhenyouhaveanestablishedconnectiontoArangoDB,thecurrentdatabasecanbechangedexplicitlyusingthedb._useDatabase()method.Thiswillswitchtothespecifieddatabase(provideditexistsandtheusercanconnecttoit).Fromthispointon,anyfollowingactioninthesameshellorconnectionwillusethespecifieddatabase,unlessotherwisespecified.

Note:Ifthedatabaseischanged,clientdriversneedtostorethecurrentdatabasenameontheirside,too.ThisisbecauseconnectionsinArangoDBdonotcontainanystateinformation.AllstateinformationiscontainedintheHTTPrequest/responsedata.

Toconnecttoaspecificdatabaseafterarangoshhasstartedusethecommanddescribedabove.Itisalsopossibletospecifyadatabasenamewheninvokingarangosh.Forthispurpose,usethecommand-lineparameter--server.database,e.g.

>arangosh--server.databasetest

Pleasenotethatcommands,actions,scriptsorAQLqueriesshouldneveraccessmultipledatabases,eveniftheyexist.TheonlyintendedandsupportedwayinArangoDBistouseonedatabaseatatimeforacommand,anaction,ascriptoraquery.Operationsstartedinonedatabasemustnotswitchthedatabaselaterandcontinueoperatinginanother.

Databases

DatabaseMethods

ThefollowingmethodsareavailabletomanagedatabasesviaJavaScript.Pleasenotethatseveralofthesemethodscanbeusedfromthe_systemdatabaseonly.

returnthedatabasenamedb._name()

Returnsthenameofthecurrentdatabaseasastring.

Examples

arangosh>require("@arangodb").db._name();

_system

returnthedatabaseiddb._id()

Returnstheidofthecurrentdatabaseasastring.

Examples

arangosh>require("@arangodb").db._id();

returnthepathtodatabasefilesdb._path()

Returnsthefilesystempathofthecurrentdatabaseasastring.

Examples

arangosh>require("@arangodb").db._path();

/tmp/arangosh_3GEDY1/tmp-21277-3063631881/data/databases/database-1

isSystem

returnthedatabasetypedb._isSystem()

Returnswhetherthecurrentlyuseddatabaseisthe_systemdatabase.Thesystemdatabasehassomespecialprivilegesandproperties,forexample,databasemanagementoperationssuchascreateordropcanonlybeexecutedfromwithinthisdatabase.Additionally,the_systemdatabaseitselfcannotbedropped.

UseDatabase

changethecurrentdatabasedb._useDatabase(name)

Changesthecurrentdatabasetothedatabasespecifiedbyname.Notethatthedatabasespecifiedbynamemustalreadyexist.

Changingthedatabasemightbedisallowedinsomecontexts,forexampleserver-sideactions(includingFoxx).

Whenperformingthiscommandfromarangosh,thecurrentcredentials(usernameandpassword)willbere-used.Thesecredentialsmightnotbevalidtoconnecttothedatabasespecifiedbyname.Additionally,thedatabaseonlybeaccessedfromcertainendpointsonly.Inthiscase,switchingthedatabasemightnotwork,andtheconnection/sessionshouldbeclosedandrestartedwithdifferentusernameandpasswordcredentialsand/orendpointdata.

ListDatabases

returnthelistofallexistingdatabasesdb._databases()

Returnsthelistofalldatabases.Thismethodcanonlybeusedfromwithinthe_systemdatabase.

CreateDatabase

createanewdatabasedb._createDatabase(name,options,users)

Createsanewdatabasewiththenamespecifiedbyname.Therearerestrictionsfordatabasenames(seeDatabaseNames).

Notethatevenifthedatabaseiscreatedsuccessfully,therewillbenochangeintothecurrentdatabasetothenewdatabase.Changingthecurrentdatabasemustexplicitlyberequestedbyusingthedb._useDatabasemethod.

Theoptionsattributecurrentlyhasnomeaningandisreservedforfutureuse.

Theoptionalusersattributecanbeusedtocreateinitialusersforthenewdatabase.Ifspecified,itmustbealistofuserobjects.Eachuserobjectcancontainthefollowingattributes:

username:theusernameasastring.Thisattributeismandatory.passwd:theuserpasswordasastring.Ifnotspecified,thenitdefaultstoanemptystring.active:abooleanflagindicatingwhethertheuseraccountshouldbeactiveornot.Thedefaultvalueistrue.extra:anoptionalJSONobjectwithextrauserinformation.ThedatacontainedinextrawillbestoredfortheuserbutnotbeinterpretedfurtherbyArangoDB.

Ifnoinitialusersarespecified,adefaultuserrootwillbecreatedwithanemptystringpassword.ThisensuresthatthenewdatabasewillbeaccessibleviaHTTPafteritiscreated.

Youcancreateusersinadatabaseifnoinitialuserisspecified.Switchintothenewdatabase(usernameandpasswordmustbeidenticaltothecurrentsession)andaddormodifyuserswiththefollowingcommands.

require("@arangodb/users").save(username,password,true);

require("@arangodb/users").update(username,password,true);

require("@arangodb/users").remove(username);

Alternatively,youcanspecifyuserdatadirectly.Forexample:

db._createDatabase("newDB",{},[{username:"newUser",passwd:"123456",active:true}])

Thosemethodscanonlybeusedfromwithinthe_systemdatabase.

DropDatabase

dropanexistingdatabasedb._dropDatabase(name)

Dropsthedatabasespecifiedbyname.Thedatabasespecifiedbynamemustexist.

Note:Droppingdatabasesisonlypossiblefromwithinthe_systemdatabase.The_systemdatabaseitselfcannotbedropped.

Databasesaredroppedasynchronously,andwillbephysicallyremovedifallclientshavedisconnectedandreferenceshavebeengarbage-collected.

Enginestatistics

retrievestatisticsrelatedtothestorageengine-rocksdbdb._engineStats()

Returnssomestatisticsrelatedtostorageengineactivity,includingfiguresaboutdatasize,cacheusage,etc.

Note:CurrentlythisonlyproducesusefuloutputfortheRocksDBengine.

NotesaboutDatabasesPleasekeepinmindthateachdatabasecontainsitsownsystemcollections,whichneedtobesetupwhenadatabaseiscreated.Thiswillmakethecreationofadatabasetakeawhile.

Replicationisconfiguredonaper-databaselevel,meaningthatanyreplicationloggingorapplyingforanewdatabasemustbeconfiguredexplicitlyafteranewdatabasehasbeencreated.

Foxxapplicationsarealsoavailableonlyinthecontextofthedatabasetheyhavebeeninstalledin.AnewdatabasewillonlyprovideaccesstothesystemapplicationsshippedwithArangoDB(thatisthewebinterfaceatthemoment)andnootherFoxxapplicationsuntiltheyareexplicitlyinstalledfortheparticulardatabase.

NotesaboutDatabases

JavaScriptInterfacetoCollectionsThisisanintroductiontoArangoDB'sinterfaceforcollectionsandhowtohandlecollectionsfromtheJavaScriptshellarangosh.ForotherlanguagesseethecorrespondinglanguageAPI.

Themostimportantcallisthecalltocreateanewcollection.

AddressofaCollection

AllcollectionsinArangoDBhaveauniqueidentifierandauniquename.ArangoDBinternallyusesthecollection'suniqueidentifiertolookupcollections.Thisidentifier,however,ismanagedbyArangoDBandtheuserhasnocontroloverit.Inordertoallowuserstousetheirownnames,eachcollectionalsohasauniquenamewhichisspecifiedbytheuser.Toaccessacollectionfromtheuserperspective,thecollectionnameshouldbeused,i.e.:

Collection

db._collection(collection-name)

Acollectioniscreatedbya"db._create"call.

Forexample:Assumethatthecollectionidentifieris7254820andthenameisdemo,thenthecollectioncanbeaccessedas:

db._collection("demo")

Ifnocollectionwithsuchanameexists,thennullisreturned.

Thereisashort-cutthatcanbeusedfornon-systemcollections:

Collectionname

db.collection-name

Thiscallwilleitherreturnthecollectionnameddb.collection-nameorcreateanewonewiththatnameandasetofdefaultproperties.

Note:Creatingacollectionontheflyusingdb.collection-nameisnotrecommendanddoesnotworkinarangosh.Tocreateanewcollection,pleaseuse

Create

db._create(collection-name)

Thiscallwillcreateanewcollectioncalledcollection-name.ThismethodisadatabasemethodandisdocumentedindetailatDatabaseMethods

Synchronousreplication

StartinginArangoDB3.0,thedistributedversionofferssynchronousreplication,whichmeansthatthereistheoptiontoreplicatealldataautomaticallywithintheArangoDBcluster.Thisisconfiguredforshardedcollectionsonapercollectionbasisbyspecifyinga"replicationfactor"whenthecollectioniscreated.Areplicationfactorofkmeansthataltogetherkcopiesofeachshardarekeptintheclusteronkdifferentservers,andarekeptinsync.Thatis,everywriteoperationisautomaticallyreplicatedonallcopies.

Thisisorganisedusingaleader/followermodel.Atalltimes,oneoftheserversholdingreplicasforashardis"theleader"andallothersare"followers",thisconfigurationisheldintheAgency(seeScalabilityfordetailsoftheArangoDBclusterarchitecture).Everywriteoperationissenttotheleaderbyoneofthecoordinators,andthenreplicatedtoallfollowersbeforetheoperationisreportedtohavesucceeded.Theleaderkeepsarecordofwhichfollowersarecurrentlyinsync.Incaseofnetworkproblemsorafailureofafollower,aleadercanandwilldropafollowertemporarilyafter3seconds,suchthatservicecanresume.Induecourse,thefollowerwillautomaticallyresynchronizewiththeleadertorestoreresilience.

Collections

Ifaleaderfails,theclusterAgencyautomaticallyinitiatesafailoverroutineafteraround15seconds,promotingoneofthefollowerstoleader.Theotherfollowers(andtheformerleader,whenitcomesback),automaticallyresynchronizewiththenewleadertorestoreresilience.Usually,thiswholefailoverprocedurecanbehandledtransparentlyforthecoordinator,suchthattheusercodedoesnotevenseeanerrormessage.

Obviously,thisfaulttolerancecomesatacostofincreasedlatency.Eachwriteoperationneedsanadditionalnetworkroundtripforthesynchronousreplicationofthefollowers,butallreplicationoperationstoallfollowershappenconcurrently.Thisis,whythedefaultreplicationfactoris1,whichmeansnoreplication.

Fordetailsonhowtoswitchonsynchronousreplicationforacollection,seethedatabasemethoddb._create(collection-name)inthesectionaboutDatabaseMethods.

Collections

CollectionMethods

dropsacollectioncollection.drop(options)

Dropsacollectionandallitsindexesanddata.Inordertodropasystemcollection,anoptionsobjectwithattributeisSystemsettotruemustbespecified.

Note:droppingacollectioninacluster,whichisprototypeforsharinginothercollectionsisprohibited.Inordertobeabletodropsuchacollection,alldependentcollectionsmustbedroppedfirst.

Examples

arangosh>col=db.example;

[ArangoCollection16878,"example"(typedocument,statusloaded)]

arangosh>col.drop();

arangosh>col;

[ArangoCollection16878,"example"(typedocument,statusdeleted)]

arangosh>col=db._example;

[ArangoCollection16889,"_example"(typedocument,statusloaded)]

arangosh>col.drop({isSystem:true});

arangosh>col;

[ArangoCollection16889,"_example"(typedocument,statusdeleted)]

Truncate

truncatesacollectioncollection.truncate()

Truncatesacollection,removingalldocumentsbutkeepingallitsindexes.

Examples

Truncatesacollection:

arangosh>col.save({"Hello":"World"});

arangosh>col.count();

arangosh>col.truncate();

showexecutionresults

Properties

getsorsetsthepropertiesofacollectioncollection.properties()Returnsanobjectcontainingallcollectionproperties.

waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheMMFilesstorageengineonly.isVolatile:IftruethenthecollectiondatawillbekeptinmemoryonlyandArangoDBwillnotwriteorsyncthedatatodisk.ThisoptionismeaningfulfortheMMFilesstorageengineonly.keyOptions(optional)additionaloptionsforkeygeneration.ThisisaJSONarraycontainingthefollowingattributes(note:someoftheattributesareoptional):

type:thetypeofthekeygeneratorusedforthecollection.allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,

CollectionMethods

thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeofdocumentsisconsideredanerror.increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.

indexBuckets:numberofbucketsintowhichindexesusingahashtablearesplit.Thedefaultis16andthisnumberhastobeapowerof2andlessthanorequalto1024.ThisoptionismeaningfulfortheMMFilesstorageengineonly.Forverylargecollectionsoneshouldincreasethistoavoidlongpauseswhenthehashtablehastobeinitiallybuiltorresized,sincebucketsareresizedindividuallyandcanbeinitiallybuiltinparallel.Forexample,64mightbeasensiblevalueforacollectionwith100000000documents.Currently,onlytheedgeindexrespectsthisvalue,butotherindextypesmightfollowinfutureArangoDBversions.Changes(seebelow)areappliedwhenthecollectionisloadedthenexttime.Inaclustersetup,theresultwillalsocontainthefollowingattributes:numberOfShards:thenumberofshardsofthecollection.shardKeys:containsthenamesofdocumentattributesthatareusedtodeterminethetargetshardfordocuments.replicationFactor:determineshowmanycopiesofeachshardarekeptondifferentDBServers.collection.properties(properties)Changesthecollectionproperties.propertiesmustbeanobjectwithoneormoreofthefollowingattribute(s):waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheMMFilesstorageengineonly.indexBuckets:Seeabove,changesareonlyappliedwhenthecollectionisloadedthenexttime.ThisoptionismeaningfulfortheMMFilesstorageengineonly.replicationFactor:ChangethenumberofshardcopieskeptondifferentDBServers,validvaluesareintegernumbersintherangeof1-10(Clusteronly)Note:itisnotpossibletochangethejournalsizeafterthejournalordatafilehasbeencreated.Changingthisparameterwillonlyeffectnewlycreatedjournals.Alsonotethatyoucannotlowerthejournalsizetolessthensizeofthelargestdocumentalreadystoredinthecollection.Note:someothercollectionproperties,suchastype,isVolatile,orkeyOptionscannotbechangedoncethecollectioniscreated.

Examples

Readallproperties

arangosh>db.example.properties();

showexecutionresultsChangeaproperty

arangosh>db.example.properties({waitForSync:true});

Figures

returnsthefiguresofacollectioncollection.figures()

Returnsanobjectcontainingstatisticsaboutthecollection.Note:Retrievingthefigureswillalwaysloadthecollectionintomemory.

alive.count:Thenumberofcurrentlyactivedocumentsinalldatafilesandjournalsofthecollection.Documentsthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.alive.size:Thetotalsizeinbytesusedbyallactivedocumentsofthecollection.Documentsthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.dead.count:Thenumberofdeaddocuments.Thisincludesdocumentversionsthathavebeendeletedorreplacedbyanewerversion.Documentsdeletedorreplacedthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.dead.size:Thetotalsizeinbytesusedbyalldeaddocuments.dead.deletion:Thetotalnumberofdeletionmarkers.Deletionmarkersonlycontainedinthewrite-aheadlogarenotreportinginthisfigure.datafiles.count:Thenumberofdatafiles.datafiles.fileSize:Thetotalfilesizeofdatafiles(inbytes).journals.count:Thenumberofjournalfiles.journals.fileSize:Thetotalfilesizeofthejournalfiles(inbytes).

CollectionMethods

compactors.count:Thenumberofcompactorfiles.compactors.fileSize:Thetotalfilesizeofthecompactorfiles(inbytes).shapefiles.count:Thenumberofshapefiles.Thisvalueisdeprecatedandkeptforcompatibilityreasonsonly.Thevaluewillalwaysbe0sinceArangoDB2.0andhigher.shapefiles.fileSize:Thetotalfilesizeoftheshapefiles.Thisvalueisdeprecatedandkeptforcompatibilityreasonsonly.Thevaluewillalwaysbe0inArangoDB2.0andhigher.shapes.count:Thetotalnumberofshapesusedinthecollection.Thisincludesshapesthatarenotinuseanymore.Shapesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.shapes.size:Thetotalsizeofallshapes(inbytes).Thisincludesshapesthatarenotinuseanymore.Shapesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.attributes.count:Thetotalnumberofattributesusedinthecollection.Note:thevalueincludesdataofattributesthatarenotinuseanymore.Attributesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.attributes.size:Thetotalsizeoftheattributedata(inbytes).Note:thevalueincludesdataofattributesthatarenotinuseanymore.Attributesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.indexes.count:Thetotalnumberofindexesdefinedforthecollection,includingthepre-definedindexes(e.g.primaryindex).indexes.size:Thetotalmemoryallocatedforindexesinbytes.lastTick:Thetickofthelastmarkerthatwasstoredinajournalofthecollection.Thismightbe0ifthecollectiondoesnotyethaveajournal.uncollectedLogfileEntries:Thenumberofmarkersinthewrite-aheadlogforthiscollectionthathavenotbeentransferredtojournalsordatafiles.documentReferences:ThenumberofreferencestodocumentsindatafilesthatJavaScriptcodecurrentlyholds.Thisinformationcanbeusedfordebuggingcompactionandunloadissues.waitingFor:Anoptionalstringvaluethatcontainsinformationaboutwhichobjecttypeisattheheadofthecollection'scleanupqueue.Thisinformationcanbeusedfordebuggingcompactionandunloadissues.compactionStatus.time:Thepointintimethecompactionforthecollectionwaslastexecuted.Thisinformationcanbeusedfordebuggingcompactionissues.compactionStatus.message:Theactionthatwasperformedwhenthecompactionwaslastrunforthecollection.Thisinformationcanbeusedfordebuggingcompactionissues.

Note:collectiondatathatarestoredinthewrite-aheadlogonlyarenotreportedintheresults.Whenthewrite-aheadlogiscollected,documentsmightbeaddedtojournalsanddatafilesofthecollection,whichmaymodifythefiguresofthecollection.AlsonotethatwaitingForandcompactionStatusmaybeemptywhencalledonacoordinatorinacluster.

Additionally,thefilesizesofcollectionandindexparameterJSONfilesarenotreported.Thesefilesshouldnormallyhaveasizeofafewbyteseach.PleasealsonotethatthefileSizevaluesarereportedinbytesandreflectthelogicalfilesizes.Somefilesystemsmayuseoptimisations(e.g.sparsefiles)sothattheactualphysicalfilesizeissomewhatdifferent.Directoriesandsub-directoriesmayalsorequirespaceinthefilesystem,butthisspaceisnotreportedinthefileSizeresults.

Thatmeansthatthefiguresreporteddonotreflecttheactualdiskusageofthecollectionwith100%accuracy.TheactualdiskusageofacollectionisnormallyslightlyhigherthanthesumofthereportedfileSizevalues.StillthesumofthefileSizevaluescanstillbeusedasalowerboundapproximationofthediskusage.

Examples

arangosh>db.demo.figures()

loadsacollectioncollection.load()

Loadsacollectionintomemory.

Note:clustercollectionsareloadedatalltimes.

Examples

CollectionMethods

arangosh>col.load();

arangosh>col;

Revision

returnstherevisionidofacollectioncollection.revision()

Returnstherevisionidofthecollection

Therevisionidisupdatedwhenthedocumentdataismodified,eitherbyinserting,deleting,updatingorreplacingdocumentsinit.

Therevisionidofacollectioncanbeusedbyclientstocheckwhetherdatainacollectionhaschangedorifitisstillunmodifiedsinceapreviousfetchoftherevisionid.

Therevisionidreturnedisastringvalue.Clientsshouldtreatthisvalueasanopaquestring,andonlyuseitforequality/non-equalitycomparisons.

returnsthephysicalpathofthecollectioncollection.path()

Thepathoperationreturnsastringwiththephysicalstoragepathforthecollectiondata.

Note:thismethodwillreturnnothingmeaningfulinacluster.Inasingle-serverArangoDB,thismethodwillonlyreturnmeaningfuldatafortheMMFilesengine.

Checksum

calculatesachecksumforthedatainacollectioncollection.checksum(withRevisions,withData)

Thechecksumoperationcalculatesanaggregatehashvalueforalldocumentkeyscontainedincollectioncollection.

IftheoptionalargumentwithRevisionsissettotrue,thentherevisionidsofthedocumentsarealsoincludedinthehashcalculation.

IftheoptionalargumentwithDataissettotrue,thenalluser-defineddocumentattributesarealsochecksummed.Includingthedocumentdatainchecksummingwillmakethecalculationslower,butismoreaccurate.

ThechecksumcalculationalgorithmchangedinArangoDB3.0,sochecksumsfrom3.0andearlierversionsforthesamedatawilldiffer.

Note:thismethodisnotavailableinacluster.

Unload

unloadsacollectioncollection.unload()

Startsunloadingacollectionfrommemory.Notethatunloadingisdeferreduntilallqueryhavefinished.

Note:clustercollectionscannotbeunloaded.

Examples

arangosh>col.unload();

arangosh>col;

[ArangoCollection7475,"example"(typedocument,statusunloaded)]

Rename

renamesacollectioncollection.rename(new-name)

CollectionMethods

Renamesacollectionusingthenew-name.Thenew-namemustnotalreadybeusedforadifferentcollection.new-namemustalsobeavalidcollectionname.Formoreinformationonvalidcollectionnamespleaserefertothenamingconventions.

Ifrenamingfailsforanyreason,anerroristhrown.Ifrenamingthecollectionsucceeds,thenthecollectionisalsorenamedinallgraphdefinitionsinsidethe_graphscollectioninthecurrentdatabase.

Note:thismethodisnotavailableinacluster.

Examples

arangosh>c=db.example;

arangosh>c.rename("better-example");

arangosh>c;

[ArangoCollection17116,"better-example"(typedocument,statusloaded)]

Rotate

rotatesthecurrentjournalofacollectioncollection.rotate()

Rotatesthecurrentjournalofacollection.Thisoperationmakesthecurrentjournalofthecollectionaread-onlydatafilesoitmaybecomeacandidateforgarbagecollection.Ifthereiscurrentlynojournalavailableforthecollection,theoperationwillfailwithanerror.

Note:thismethodisspecificfortheMMFilesstorageengine,andthereitisnotavailableinacluster.

Note:pleasenotethatyouneedappropriateuserpermissionstoexecutethis.

TodotherenamecollectionsinfirstplaceyouneedtohaveadministrativerightsonthedatabaseTohaveaccesstotheresultingrenamedcollectionyoueitherneedtohaveaccesstoallcollectionsofthatdatabase(*)oramainsystemadministratorhastogiveyouaccesstothenewlynamedone.

CollectionMethods

DatabaseMethods

Collection

returnsasinglecollectionornulldb._collection(collection-name)

Returnsthecollectionwiththegivennameornullifnosuchcollectionexists.

db._collection(collection-identifier)

Returnsthecollectionwiththegivenidentifierornullifnosuchcollectionexists.Accessingcollectionsbyidentifierisdiscouragedforendusers.Endusersshouldaccesscollectionsusingthecollectionname.

Examples

Getacollectionbyname:

arangosh>db._collection("demo");

[ArangoCollection102,"demo"(typedocument,statusloaded)]

Getacollectionbyid:

arangosh>db._collection(123456);

[ArangoCollection123456,"demo"(typedocument,statusloaded)]

Unknowncollection:

arangosh>db._collection("unknown");

Create

createsanewdocumentoredgecollectiondb._create(collection-name)

Createsanewdocumentcollectionnamedcollection-name.Ifthecollectionnamealreadyexistsorifthenameformatisinvalid,anerroristhrown.Formoreinformationonvalidcollectionnamespleaserefertothenamingconventions.

db._create(collection-name,properties)

propertiesmustbeanobjectwiththefollowingattributes:

waitForSync(optional,defaultfalse):Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.

journalSize(optional,defaultisaconfigurationparameter:Themaximalsizeofajournalordatafile.Notethatthisalsolimitsthemaximalsizeofasingleobject.Mustbeatleast1MB.

isSystem(optional,defaultisfalse):Iftrue,createasystemcollection.Inthiscasecollection-nameshouldstartwithanunderscore.Endusersshouldnormallycreatenon-systemcollectionsonly.APIimplementorsmayberequiredtocreatesystemcollectionsinveryspecialoccasions,butnormallyaregularcollectionwilldo.

isVolatile(optional,defaultisfalse):Iftruethenthecollectiondataiskeptin-memoryonlyandnotmadepersistent.Unloadingthecollectionwillcausethecollectiondatatobediscarded.Stoppingorre-startingtheserverwillalsocausefulllossofdatainthecollection.Thecollectionitselfwillremainhowever(onlythedataisvolatile).SettingthisoptionwillmaketheresultingcollectionbeslightlyfasterthanregularcollectionsbecauseArangoDBdoesnotenforceanysynchronizationtodiskanddoesnotcalculateanyCRCchecksumsfordatafiles(astherearenodatafiles).ThisoptionismeaningfulfortheMMFilesstorageengineonly.

keyOptions(optional):additionaloptionsforkeygeneration.Ifspecified,thenkeyOptionsshouldbeaJSONobjectcontainingthefollowingattributes(note:someofthemareoptional):

type:specifiesthetypeofthekeygenerator.Thecurrentlyavailablegeneratorsaretraditionalandautoincrement.(note:autoincrementiscurrentlyonlysupportedfornon-shardedcollections)

DatabaseMethods

allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeofdocumentsisconsideredanerror.increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.

numberOfShards(optional,defaultis1):inacluster,thisvaluedeterminesthenumberofshardstocreateforthecollection.Inasingleserversetup,thisoptionismeaningless.

shardKeys(optional,defaultis["_key"]):inacluster,thisattributedetermineswhichdocumentattributesareusedtodeterminethetargetshardfordocuments.Documentsaresenttoshardsbasedonthevaluestheyhaveintheirshardkeyattributes.Thevaluesofallshardkeyattributesinadocumentarehashed,andthehashvalueisusedtodeterminethetargetshard.Notethatvaluesofshardkeyattributescannotbechangedonceset.Thisoptionismeaninglessinasingleserversetup.

Whenchoosingtheshardkeys,onemustbeawareofthefollowingrulesandlimitations:InashardedcollectionwithmorethanonesharditisnotpossibletosetupauniqueconstraintonanattributethatisnottheoneandonlyshardkeygiveninshardKeys.Thisisbecauseenforcingauniqueconstraintwouldotherwisemakeaglobalindexnecessaryorneedextensivecommunicationforeverysinglewriteoperation.Furthermore,if_keyisnottheoneandonlyshardkey,thenitisnotpossibletosetthe_keyattributewheninsertingadocument,providedthecollectionhasmorethanoneshard.Again,thisisbecausethedatabasehastoenforcetheuniqueconstraintonthe_keyattributeandthiscanonlybedoneefficientlyifthisistheonlyshardkeybydelegatingtotheindividualshards.

replicationFactor(optional,defaultis1):inacluster,thisattributedetermineshowmanycopiesofeachshardarekeptondifferentDBServers.Thevalue1meansthatonlyonecopy(nosynchronousreplication)iskept.Avalueofkmeansthatk-1replicasarekept.AnytwocopiesresideondifferentDBServers.Replicationbetweenthemissynchronous,thatis,everywriteoperationtothe"leader"copywillbereplicatedtoall"follower"replicas,beforethewriteoperationisreportedsuccessful.

Ifaserverfails,thisisdetectedautomaticallyandoneoftheserversholdingcopiestakeover,usuallywithoutanerrorbeingreported.

distributeShardsLikedistributetheshardsofthiscollectioncloningthesharddistributionofanother.

WhenusingtheEnterpriseversionofArangoDBthereplicationFactormaybesetto"satellite"makingthecollectionlocallyjoinableoneverydatabaseserver.ThisreducesthenumberofnetworkhopsdramaticallywhenusingjoinsinAQLatthecostsofreducedwriteperformanceonthesecollections.

db._create(collection-name,properties,type)

Specifiestheoptionaltypeofthecollection,itcaneitherbedocumentoredge.Ondefaultitisdocument.Insteadofgivingatypeyoucanalsousedb._createEdgeCollectionordb._createDocumentCollection.

db._create(collection-name,properties[,type],options)

Asanoptionalthird(ifthetypestringisbeingomitted)orfourthparameteryoucanspecifyanoptionaloptionsmapthatcontrolshowtheclusterwillcreatethecollection.Theseoptionsareonlyrelevantatcreationtimeandwillnotbepersisted:

waitForSyncReplication(default:true)Whenenabledtheserverwillonlyreportsuccessbacktotheclientifallreplicashavecreatedthecollection.Settofalseifyouwantfasterserverresponsesanddon'tcareaboutfullreplication.

enforceReplicationFactor(default:true)Whenenabledwhichmeanstheserverwillcheckifthereareenoughreplicasavailableatcreationtimeandbailoutotherwise.Settofalsetodisablethisextracheck.

Examples

Withdefaults:

arangosh>c=db._create("users");

arangosh>c.properties();

showexecutionresultsWithproperties:

arangosh>c=db._create("users",{waitForSync:true,

........>journalSize:1024*1204});

DatabaseMethods

arangosh>c.properties();

showexecutionresultsWithakeygenerator:

arangosh>db._create("users",

........>{keyOptions:{type:"autoincrement",offset:10,increment:5}});

arangosh>db.users.save({name:"user1"});

showexecutionresultsWithaspecialkeyoption:

arangosh>db._create("users",{keyOptions:{allowUserKeys:false}});

arangosh>db.users.save({name:"user2",_key:"myuser"});

showexecutionresultscreatesanewedgecollectiondb._createEdgeCollection(collection-name)

Createsanewedgecollectionnamedcollection-name.Ifthecollectionnamealreadyexistsanerroristhrown.ThedefaultvalueforwaitForSyncisfalse.

db._createEdgeCollection(collection-name,properties)

propertiesmustbeanobjectwiththefollowingattributes:

waitForSync(optional,defaultfalse):Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize(optional,defaultis"configurationparameter"):Themaximalsizeofajournalordatafile.Notethatthisalsolimitsthemaximalsizeofasingleobjectandmustbeatleast1MB.

createsanewdocumentcollectiondb._createDocumentCollection(collection-name)

Createsanewdocumentcollectionnamedcollection-name.Ifthedocumentnamealreadyexistsanderroristhrown.

AllCollections

returnsallcollectionsdb._collections()

Returnsallcollectionsofthegivendatabase.

Examples

arangosh>db._collections();

CollectionName

selectsacollectionfromthevocbasedb.collection-name

Returnsthecollectionwiththegivencollection-name.Ifnosuchcollectionexists,createacollectionnamedcollection-namewiththedefaultproperties.

Examples

arangosh>db.example;

DatabaseMethods

dropsacollectiondb._drop(collection)

Dropsacollectionandallitsindexesanddata.

db._drop(collection-identifier)

Dropsacollectionidentifiedbycollection-identifierwithallitsindexesanddata.Noerroristhrownifthereisnosuchcollection.

db._drop(collection-name)

Dropsacollectionnamedcollection-nameandallitsindexes.Noerroristhrownifthereisnosuchcollection.

db._drop(collection-name,options)

Inordertodropasystemcollection,onemustspecifyanoptionsobjectwithattributeisSystemsettotrue.Otherwiseitisnotpossibletodropsystemcollections.

Note:clustercollection,whichareprototypesforcollectionswithdistributeShardsLikeparameter,cannotbedropped.

Examples

Dropsacollection:

arangosh>db._drop(col);

arangosh>col;

Dropsacollectionidentifiedbyname:

arangosh>db._drop("example");

arangosh>col;

[ArangoCollection16784,"example"(typedocument,statusdeleted)]

Dropsasystemcollection

arangosh>col=db._example;

[ArangoCollection16795,"_example"(typedocument,statusloaded)]

arangosh>db._drop("_example",{isSystem:true});

arangosh>col;

[ArangoCollection16795,"_example"(typedocument,statusdeleted)]

Truncate

truncatesacollectiondb._truncate(collection)

Truncatesacollection,removingalldocumentsbutkeepingallitsindexes.

db._truncate(collection-identifier)

Truncatesacollectionidentifiedbycollection-identified.Noerroristhrownifthereisnosuchcollection.

db._truncate(collection-name)

Truncatesacollectionnamedcollection-name.Noerroristhrownifthereisnosuchcollection.

Examples

Truncatesacollection:

DatabaseMethods

arangosh>db._truncate(col);

showexecutionresultsTruncatesacollectionidentifiedbyname:

arangosh>db._truncate("example");

DatabaseMethods

DocumentsThisisanintroductiontoArangoDB'sinterfaceforworkingwithdocumentsfromtheJavaScriptshellarangoshorinJavaScriptcodeintheserver.ForotherlanguagesseethecorrespondinglanguageAPI.

BasicsandTerminology:sectiononthebasicapproachCollectionMethods:detailedAPIdescriptionforcollectionobjectsDatabaseMethods:detailedAPIdescriptionfordatabaseobjects

Documents

BasicsandTerminologyDocumentsinArangoDBareJSONobjects.Theseobjectscanbenested(toanydepth)andmaycontainlists.Eachdocumenthasauniqueprimarykeywhichidentifiesitwithinitscollection.Furthermore,eachdocumentisuniquelyidentifiedbyitsdocumenthandleacrossallcollectionsinthesamedatabase.Differentrevisionsofthesamedocument(identifiedbyitshandle)canbedistinguishedbytheirdocumentrevision.Anytransactiononlyeverseesasinglerevisionofadocument.Forexample:

"_id":"myusers/3456789",

"_key":"3456789",

"_rev":"14253647",

"firstName":"John",

"lastName":"Doe",

"address":{

"street":"RoadToNowhere1",

"city":"Gotham"

"hobbies":[

Alldocumentscontainspecialattributes:thedocumenthandleisstoredasastringin_id,thedocument'sprimarykeyin_keyandthedocumentrevisionin_rev.Thevalueofthe_keyattributecanbespecifiedbytheuserwhencreatingadocument._idand_keyvaluesareimmutableoncethedocumenthasbeencreated.The_revvalueismaintainedbyArangoDBautomatically.

DocumentHandle

Adocumenthandleuniquelyidentifiesadocumentinthedatabase.Itisastringandconsistsofthecollection'snameandthedocumentkey(_keyattribute)separatedby/.

DocumentKey

Adocumentkeyuniquelyidentifiesadocumentinthecollectionitisstoredin.Itcanandshouldbeusedbyclientswhenspecificdocumentsarequeried.Thedocumentkeyisstoredinthe_keyattributeofeachdocument.ThekeyvaluesareautomaticallyindexedbyArangoDBinacollection'sprimaryindex.Thuslookingupadocumentbyitskeyisafastoperation.The_keyvalueofadocumentisimmutableoncethedocumenthasbeencreated.Bydefault,ArangoDBwillauto-generateadocumentkeyifno_keyattributeisspecified,andusetheuser-specified_keyotherwise.Thegenerated_keyisguaranteedtobeuniqueinthecollectionitwasgeneratedfor.Thisalsoappliestoshardedcollectionsinacluster.Itcan'tbeguaranteedthatthe_keyisuniquewithinadatabaseoracrossawholenodeorinstancehowever.

Thisbehaviorcanbechangedonaper-collectionlevelbycreatingcollectionswiththekeyOptionsattribute.

UsingkeyOptionsitispossibletodisallowuser-specifiedkeyscompletely,ortoforceaspecificregimeforauto-generatingthe_keyvalues.

DocumentRevision

AsArangoDBsupportsMVCC(MultipleVersionConcurrencyControl),documentscanexistinmorethanonerevision.ThedocumentrevisionistheMVCCtokenusedtospecifyaparticularrevisionofadocument(identifiedbyits_id).Itisastringvaluethatcontained(uptoArangoDB3.0)anintegernumberandisuniquewithinthelistofdocumentrevisionsforasingledocument.InArangoDB>=3.1the_revstringsareinfacttimestamps.TheyusethelocalclockoftheDBserverthatactuallywritesthedocumentandhavemillisecondaccuracy.Actually,a"HybridLogicalClock"isused(forthisconceptseethispaper).

Withinonesharditisguaranteedthattwodifferentdocumentrevisionshaveadifferent_revstring,eveniftheyarewritteninthesamemillisecond,andthatthesestampsareascending.

Notehoweverthatdifferentserversinyourclustermighthaveaclockskew,andthereforebetweendifferentshardsorevenbetweendifferentcollectionsthetimestampsarenotguaranteedtobecomparable.

TheHybridLogicalClockfeaturedoesonethingtoaddressthisissue:WheneveramessageissentfromsomeserverAinyourclustertoanotheroneB,itisensuredthatanytimestamptakenonBafterthemessagehasarrivedisgreaterthananytimestamptakenonAbeforethemessagewassent.Thisensuresthatifthereissome"causality"betweeneventsondifferentservers,timestampsincreasefromcausetoeffect.Adirectconsequenceofthisisthatsometimesaserverhastotaketimestampsthatseemtocomefromthefutureofitsownclock.Itwillhoweverstillproduceeverincreasingtimestamps.Iftheclockskewissmall,thenyourtimestampswillrelativelyaccuratelydescribethetimewhenthedocumentrevisionwasactuallywritten.

ArangoDBuses64bitunsignedintegervaluestomaintaindocumentrevisionsinternally.Atthisstageweintentionallydonotdocumenttheexactformatoftherevisionvalues.Whenreturningdocumentrevisionstoclients,ArangoDBwillputthemintoastringtoensuretherevisionisnotclippedbyclientsthatdonotsupportbigintegers.ClientsshouldtreattherevisionreturnedbyArangoDBasanopaquestringwhentheystoreoruseitlocally.ThiswillallowArangoDBtochangetheformatofrevisionslaterifthisshouldberequired(ashashappenedwith3.1withtheHybridLogicalClock).Clientscanuserevisionstoperformsimpleequality/non-equalitycomparisons(e.g.tocheckwhetheradocumenthaschangedornot),buttheyshouldnotuserevisionidstoperformgreater/lessthancomparisonswiththemtocheckifadocumentrevisionisolderthanoneanother,evenifthismightworkforsomecases.

Documentrevisionscanbeusedtoconditionallyquery,update,replaceordeletedocumentsinthedatabase.Inordertofindaparticularrevisionofadocument,youneedthedocumenthandleorkey,andthedocumentrevision.

MultipleDocumentsinasingleCommand

BeginningwithArangoDB3.0thebasicdocumentAPIhasbeenextendedtohandlenotonlysingledocumentsbutmultipledocumentsinasinglecommand.Thisiscrucialforperformance,inparticularintheclustersituation,inwhichasinglerequestcaninvolvemultiplenetworkhopswithinthecluster.Anotheradvantageisthatitreducestheoverheadofindividualnetworkroundtripsbetweentheclientandtheserver.ThegeneralideatoperformmultipledocumentoperationsinasinglecommandistouseJSONarraysofobjectsintheplaceofasingledocument.Asaconsequence,documentkeys,handlesandrevisionsforpreconditionshavetobesuppliedembeddedintheindividualdocumentsgiven.Multipledocumentoperationsarerestrictedtoasingledocumentoredgecollection.SeetheAPIdescriptionsforcollectionobjectsfordetails.NotethattheAPIfordatabaseobjectsdonotoffertheseoperations.

CollectionMethods

collection.all()

Fetchesalldocumentsfromacollectionandreturnsacursor.YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.

Examples

UsetoArraytogetalldocumentsatonce:

arangosh>db.five.save({name:"one"});

arangosh>db.five.save({name:"two"});

arangosh>db.five.save({name:"three"});

arangosh>db.five.save({name:"four"});

arangosh>db.five.save({name:"five"});

arangosh>db.five.all().toArray();

showexecutionresultsUselimittorestrictthedocuments:

arangosh>db.five.save({name:"one"});

arangosh>db.five.save({name:"two"});

arangosh>db.five.save({name:"three"});

arangosh>db.five.save({name:"four"});

arangosh>db.five.save({name:"five"});

arangosh>db.five.all().limit(2).toArray();

Querybyexamplecollection.byExample(example)

Fetchesalldocumentsfromacollectionthatmatchthespecifiedexampleandreturnsacursor.

YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.

Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.Ifyouuse

{"a":{"c":1}}

asexample,thenyouwillfindalldocuments,suchthattheattributeacontainsadocumentoftheform{c:1}.Forexamplethedocument

{"a":{"c":1},"b":1}

willmatch,butthedocument

{"a":{"c":1,"b":1}}

willnot.

However,ifyouuse

{"a.c":1}

CollectionMethods

thenyouwillfindalldocuments,whichcontainasub-documentinathathasanattributecofvalue1.Boththefollowingdocuments

{"a":{"c":1},"b":1}

{"a":{"c":1,"b":1}}

willmatch.

collection.byExample(path1,value1,...)

Asalternativeyoucansupplyanarrayofpathsandvalues.

Examples

arangosh>db.users.save({name:"Gerhard"});

arangosh>db.users.save({name:"Helmut"});

arangosh>db.users.save({name:"Angela"});

arangosh>db.users.all().toArray();

arangosh>db.users.byExample({"_id":"users/20"}).toArray();

arangosh>db.users.byExample({"name":"Gerhard"}).toArray();

arangosh>db.users.byExample({"name":"Helmut","_id":"users/15"}).toArray();

showexecutionresultsUsenexttoloopoveralldocuments:

arangosh>db.users.save({name:"Gerhard"});

arangosh>db.users.save({name:"Helmut"});

arangosh>db.users.save({name:"Angela"});

arangosh>vara=db.users.byExample({"name":"Angela"});

arangosh>while(a.hasNext())print(a.next());

FirstExample

collection.firstExample(example)

Returnssomedocumentofacollectionthatmatchesthespecifiedexample.Ifnosuchdocumentexists,nullwillbereturned.Theexamplehastobespecifiedaspathsandvalues.SeebyExamplefordetails.

collection.firstExample(path1,value1,...)

Asalternativeyoucansupplyanarrayofpathsandvalues.

Examples

arangosh>db.users.firstExample("name","Angela");

collection.range(attribute,left,right)

CollectionMethods

Returnsalldocumentsfromacollectionsuchthattheattributeisgreaterorequalthanleftandstrictlylessthanright.

Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.

Note:therangesimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionwithinaspecificrangeistouseanAQLqueryasfollows:

FORdocIN@@collection

FILTERdoc.value>=@left&&doc.value<@right

LIMIT@skip,@limit

RETURNdoc

Examples

arangosh>db.old.ensureIndex({type:"skiplist",fields:["age"]});

arangosh>db.old.save({age:15});

arangosh>db.old.range("age",10,30).toArray();

Closedrange

collection.closedRange(attribute,left,right)

Returnsalldocumentsofacollectionsuchthattheattributeisgreaterorequalthanleftandlessorequalthanright.

Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.

Note:theclosedRangesimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionwithinaspecificrangeistouseanAQLqueryasfollows:

FILTERdoc.value>=@left&&doc.value<=@right

LIMIT@skip,@limit

RETURNdoc

Examples

arangosh>db.old.ensureIndex({type:"skiplist",fields:["age"]});

arangosh>db.old.closedRange("age",10,30).toArray();

collection.any()

Returnsarandomdocumentfromthecollectionornullifnoneexists.

Note:thismethodisexpensivewhenusingtheRocksDBstorageengine.

CollectionMethods

collection.count()

Returnsthenumberoflivingdocumentsinthecollection.

Examples

arangosh>db.users.count();

toArray

collection.toArray()

Convertsthecollectionintoanarrayofdocuments.NeverusethiscallinaproductionenvironmentasitwillbasicallycreateacopyofyourcollectioninRAMwhichwilluseresourcesdependingonthenumberandsizeofthedocumentsinyourcollecion.

Documentcollection.document(object)

Thedocumentmethodfindsadocumentgivenanobjectobjectcontainingthe_idor_keyattribute.Themethodreturnsthedocumentifitcanbefound.Ifbothattributesaregiven,the_idtakesprecedence,itisanerror,ifthecollectionpartofthe_iddoesnotmatchthecollection.

Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.Anerrorisalsothrownifnodocumentexistswiththegiven_idor_keyvalue.

Pleasenotethatifthemethodisexecutedonthearangodserver(e.g.frominsideaFoxxapplication),animmutabledocumentobjectwillbereturnedforperformancereasons.Itisnotpossibletochangeattributesofthisimmutableobject.Toupdateorpatchthereturneddocument,itneedstobecloned/copiedintoaregularJavaScriptobjectfirst.Thisisnotnecessaryifthedocumentmethodiscalledfromoutofarangoshorfromanyotherclient.

collection.document(document-handle)

Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.Norevisioncanbespecifiedinthiscase.

collection.document(document-key)

Asbefore.Insteadofobjectadocument-keycanbepassedasfirstargument.

collection.document(array)

Thisvariantallowstoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifdocumentwouldhavebeencalledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.

Examples

Returnsthedocumentforadocument-handle:

arangosh>db.example.document("example/2873916");

showexecutionresultsReturnsthedocumentforadocument-key:

arangosh>db.example.document("2873916");

showexecutionresultsReturnsthedocumentforanobject:

arangosh>db.example.document({_id:"example/2873916"});

CollectionMethods

showexecutionresultsReturnsthedocumentforanarrayoftwokeys:

arangosh>db.example.document(["2873916","2873917"]);

showexecutionresultsAnerrorisraisedifthedocumentisunknown:

[ArangoError1202:documentnotfound]

Anerrorisraisedifthehandleisinvalid:

arangosh>db.example.document("");

[ArangoError1205:illegaldocumenthandle]

Changesin3.0from2.8:

documentcannowquerymultipledocumentswithonecall.

Exists

checkswhetheradocumentexistscollection.exists(object)

Theexistsmethoddetermineswhetheradocumentexistsgivenanobjectobjectcontainingthe_idor_keyattribute.Ifbothattributesaregiven,the_idtakesprecedence,itisanerror,ifthecollectionpartofthe_iddoesnotmatchthecollection.

Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.

Insteadofreturningthefounddocumentoranerror,thismethodwillonlyreturnanobjectwiththeattributes_id,_keyand_rev,orfalseifnodocumentwiththegiven_idor_keyexists.Itcanthusbeusedforeasyexistencechecks.

Thismethodwillthrowanerrorifusedimproperly,e.g.whencalledwithanon-documenthandle,anon-document,orwhenacross-collectionrequestisperformed.

collection.exists(document-handle)

Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.

collection.exists(document-key)

Asbefore.Insteadofobjectadocument-keycanbepassedasfirstargument.

collection.exists(array)

Thisvariantallowstoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifexistswouldhavebeencalledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,theoperationstopsimmediatelyreturningonlyanerrorobject.

Inthecaseofarevisionmismatchexistsnowthrowsanerrorinsteadofsimplyreturningfalse.Thisistomakeitpossibletotellthedifferencebetweenarevisionmismatchandanon-existingdocument.

existscannowquerymultipledocumentswithonecall.

LookupByKeyscollection.documents(keys)

CollectionMethods

Looksupthedocumentsinthespecifiedcollectionusingthearrayofkeysprovided.Alldocumentsforwhichamatchingkeywasspecifiedinthekeysarrayandthatexistinthecollectionwillbereturned.Keysforwhichnodocumentcanbefoundintheunderlyingcollectionareignored,andnoexceptionwillbethrownforthem.

Thismethodisdeprecatedinfavourofthearrayvariantofdocument.

Examples

arangosh>keys=[];

arangosh>for(vari=0;i<10;++i){

........>db.example.insert({_key:"test"+i,value:i});

........>keys.push("test"+i);

........>}

arangosh>db.example.documents(keys);

Insert

collection.insert(data)

Createsanewdocumentinthecollectionfromthegivendata.Thedatamustbeanobject.Theattributes_idand_revareignoredandareautomaticallygenerated.Auniquevaluefortheattribute_keywillbeautomaticallygeneratedifnotspecified.Ifspecified,theremustnotbeadocumentwiththegiven_keyinthecollection.

Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleofthenewlycreateddocument,theattribute_keythedocumentkeyandtheattribute_revcontainsthedocumentrevision.

collection.insert(data,options)

Createsanewdocumentinthecollectionfromthegivendataasabove.Theoptionaloptionsparametermustbeanobjectandcanbeusedtospecifythefollowingoptions:

waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.silent:Ifthisflagissettotrue,themethoddoesnotreturnanyoutput.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.

Note:sinceArangoDB2.2,insertisanaliasforsave.

collection.insert(array)

collection.insert(array,options)

Thesetwovariantsallowtoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifinsertwouldhavebeencalledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.

TheoptionssilentandreturnNewarenew.Themethodcannowinsertmultipledocumentswithonecall.

Examples

arangosh>db.example.insert({Hello:"World"});

arangosh>db.example.insert({Hello:"World"},{waitForSync:true});

arangosh>db.example.insert([{Hello:"World"},{Hello:"there"}])

CollectionMethods

arangosh>db.example.insert([{Hello:"World"},{}],{waitForSync:true});

Replace

collection.replace(selector,data)

Replacesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbeadocumentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenreplacedwiththedatagivenassecondargument.Anyattribute_id,_keyor_revindataisignored.

Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleoftheupdateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainstherevisionoftheold(nowreplaced)document.

Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,thereisaconflict,andanerroristhrown.

collection.replace(selector,data,options)

Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:

waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.

collection.replace(document-handle,data)

collection.replace(document-handle,data,options)

Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.

collection.replace(document-key,data)

collection.replace(document-key,data,options)

Asbefore.Insteadofselectoradocument-keycanbepassedasfirstargument.Norevisionpreconditionistested.

collection.replace(selectorarray,dataarray)

collection.replace(selectorarray,dataarray,options)

Thesetwovariantsallowtoperformtheoperationonawholearrayofselector/datapairs.Thetwoarraysgivenasselectorarrayanddataarraymusthavethesamelength.Thebehaviorisexactlyasifreplacewouldhavebeencalledonallrespectivemembersofthetwoarraysandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.

Examples

Createandupdateadocument:

arangosh>a1=db.example.insert({a:1});

arangosh>a2=db.example.replace(a1,{a:2});

showexecutionresultsUseadocumenthandle:

CollectionMethods

arangosh>a2=db.example.replace("example/3903044",{a:2});

Theoptionssilent,returnNewandreturnOldarenew.Themethodcannowreplacemultipledocumentswithonecall.

Update

collection.update(selector,data)

Updatesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbeadocumentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenpatchedwiththedatagivenassecondargument.Anyattribute_id,_keyor_revindataisignored.

Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleoftheupdateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainstherevisionoftheold(nowupdated)document.

collection.update(selector,data,options)

waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.keepNull:TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesarestoredinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnullvalueswillberemovedfromthetargetdocument.mergeObjects:Controlswhetherobjects(notarrays)willbemergedifpresentinboththeexistingandthepatchdocument.Ifsettofalse,thevalueinthepatchdocumentwilloverwritetheexistingdocument'svalue.Ifsettotrue,objectswillbemerged.Thedefaultistrue.

collection.update(document-handle,data)

collection.update(document-handle,data,options)

collection.update(document-key,data)

collection.update(document-key,data,options)

Asbefore.Insteadofselectoradocument-keycanbepassedasfirstargument.Norevisionpreconditionistested.

collection.update(selectorarray,dataarray)

collection.update(selectorarray,dataarray,options)

Thesetwovariantsallowtoperformtheoperationonawholearrayofselector/datapairs.Thetwoarraysgivenasselectorarrayanddataarraymusthavethesamelength.Thebehaviorisexactlyasifupdatewouldhavebeencalledonallrespectivemembersofthetwoarraysandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.

Examples

CollectionMethods

arangosh>a1=db.example.insert({"a":1});

arangosh>a2=db.example.update(a1,{"b":2,"c":3});

arangosh>a3=db.example.update(a1,{"d":4});

arangosh>a4=db.example.update(a2,{"e":5,"f":6});

arangosh>db.example.document(a4);

arangosh>a5=db.example.update(a4,{"a":1,c:9,e:42});

showexecutionresultsUseadocumenthandle:

arangosh>a1=db.example.insert({"a":1});

arangosh>a2=db.example.update("example/18612115",{"x":1,"y":2});

showexecutionresultsUsethekeepNullparametertoremoveattributeswithnullvalues:

arangosh>db.example.insert({"a":1});

arangosh>db.example.update("example/19988371",

........>{"b":null,"c":null,"d":3});

arangosh>db.example.update("example/19988371",{"a":null},false,false);

arangosh>db.example.update("example/19988371",

........>{"b":null,"c":null,"d":null},false,false);

showexecutionresultsPatchingarrayvalues:

arangosh>db.example.insert({"a":{"one":1,"two":2,"three":3},

........>"b":{}});

arangosh>db.example.update("example/20774803",{"a":{"four":4},

........>"b":{"b1":1}});

arangosh>db.example.update("example/20774803",{"a":{"one":null},

........>"b":null},

........>false,false);

Theoptionssilent,returnNewandreturnOldarenew.Themethodcannowupdatemultipledocumentswithonecall.

Remove

collection.remove(selector)

Removesadocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbeadocumentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenremoved.

Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleoftheremoveddocument,theattribute_revcontainsthedocumentrevisionoftheremoveddocument.

CollectionMethods

collection.remove(selector,options)

waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.

collection.remove(document-handle)

collection.remove(document-handle,options)

Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisioncheckisperformed.

collection.remove(document-key)

collection.remove(document-handle,options)

collection.remove(selectorarray)

collection.remove(selectorarray,options)

Thesetwovariantsallowtoperformtheoperationonawholearrayofselectors.Thebehaviorisexactlyasifremovewouldhavebeencalledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.

Examples

Removeadocument:

arangosh>db.example.remove(a1);

showexecutionresultsRemoveadocumentwithaconflict:

arangosh>db.example.remove(a1);

arangosh>db.example.remove(a1,true);

Themethodnowreturnsnotonlytruebutinformationabouttheremoveddocument(s).TheoptionssilentandreturnOldarenew.Themethodcannowremovemultipledocumentswithonecall.

RemoveByKeys

collection.removeByKeys(keys)

CollectionMethods

Looksupthedocumentsinthespecifiedcollectionusingthearrayofkeysprovided,andremovesalldocumentsfromthecollectionwhosekeysarecontainedinthekeysarray.Keysforwhichnodocumentcanbefoundintheunderlyingcollectionareignored,andnoexceptionwillbethrownforthem.

Themethodwillreturnanobjectcontainingthenumberofremoveddocumentsintheremovedsub-attribute,andthenumberofnot-removed/ignoreddocumentsintheignoredsub-attribute.

Thismethodisdeprecatedinfavourofthearrayvariantofremove.

Examples

arangosh>keys=[];

arangosh>for(vari=0;i<10;++i){

........>db.example.insert({_key:"test"+i,value:i});

........>keys.push("test"+i);

........>}

arangosh>db.example.removeByKeys(keys);

RemoveByExamplecollection.removeByExample(example)

Removesalldocumentsmatchinganexample.

collection.removeByExample(document,waitForSync)

TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentdeletionoperationtodiskevenincasethatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.

collection.removeByExample(document,waitForSync,limit)

Theoptionallimitparametercanbeusedtorestrictthenumberofremovalstothespecifiedvalue.Iflimitisspecifiedbutlessthanthenumberofdocumentsinthecollection,itisundefinedwhichdocumentsareremoved.

Examples

arangosh>db.example.removeByExample({Hello:"world"});

ReplaceByExample

collection.replaceByExample(example,newValue)

Replacesalldocumentsmatchinganexamplewithanewdocumentbody.TheentiredocumentbodyofeachdocumentmatchingtheexamplewillbereplacedwithnewValue.Thedocumentmeta-attributes_id,_keyand_revwillnotbereplaced.

collection.replaceByExample(document,newValue,waitForSync)

TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentreplacementoperationtodiskevenincasethatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.

collection.replaceByExample(document,newValue,waitForSync,limit)

Theoptionallimitparametercanbeusedtorestrictthenumberofreplacementstothespecifiedvalue.Iflimitisspecifiedbutlessthanthenumberofdocumentsinthecollection,itisundefinedwhichdocumentsarereplaced.

CollectionMethods

Examples

arangosh>db.example.save({Hello:"world"});

arangosh>db.example.replaceByExample({Hello:"world"},{Hello:"mars"},false,5);

UpdateByExamplecollection.updateByExample(example,newValue)

Partiallyupdatesalldocumentsmatchinganexamplewithanewdocumentbody.SpecificattributesinthedocumentbodyofeachdocumentmatchingtheexamplewillbeupdatedwiththevaluesfromnewValue.Thedocumentmeta-attributes_id,_keyand_revcannotbeupdated.

Partialupdatecouldalsobeusedtoappendnewfields,iftherewerenooldfieldwithsamename.

collection.updateByExample(document,newValue,keepNull,waitForSync)

TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesarestoredinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnullvalueswillberemovedfromthetargetdocument.

TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentreplacementoperationtodiskevenincasethatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.

collection.updateByExample(document,newValue,keepNull,waitForSync,limit)

Theoptionallimitparametercanbeusedtorestrictthenumberofupdatestothespecifiedvalue.Iflimitisspecifiedbutlessthanthenumberofdocumentsinthecollection,itisundefinedwhichdocumentsareupdated.

collection.updateByExample(document,newValue,options)

Usingthisvariant,theoptionsfortheoperationcanbepassedusinganobjectwiththefollowingsub-attributes:

keepNullwaitForSynclimitmergeObjects

Examples

arangosh>db.example.save({Hello:"world",foo:"bar"});

arangosh>db.example.updateByExample({Hello:"world"},{Hello:"foo",World:"bar"},

false);

arangosh>db.example.byExample({Hello:"foo"}).toArray()

Collectiontypecollection.type()

Returnsthetypeofacollection.Possiblevaluesare:

2:documentcollection3:edgecollection

GettheVersionofArangoDBdb._version()

CollectionMethods

Returnstheserverversionstring.Notethatthisisnottheversionofthedatabase.

Examples

arangosh>require("@arangodb").db._version();

3.4.milestone0

Edgesarenormaldocumentsthatalwayscontaina_fromanda_toattribute.Therefore,youcanusethedocumentmethodstooperateonedges.Thefollowingmethods,however,arespecifictoedges.

edge-collection.edges(vertex)

Theedgesoperatorfindsalledgesstartingfrom(outbound)orendingin(inbound)vertex.

edge-collection.edges(vertices)

Theedgesoperatorfindsalledgesstartingfrom(outbound)orendingin(inbound)adocumentfromvertices,whichmustbealistofdocumentsordocumenthandles.

arangosh>db._create("vertex");

arangosh>db._createEdgeCollection("relation");

arangosh>varmyGraph={};

arangosh>myGraph.v1=db.vertex.insert({name:"vertex1"});

arangosh>myGraph.e1=db.relation.insert(myGraph.v1,myGraph.v2,

........>{label:"knows"});

arangosh>db._document(myGraph.e1);

arangosh>db.relation.edges(myGraph.e1._id);

showexecutionresultsedge-collection.inEdges(vertex)

Theedgesoperatorfindsalledgesendingin(inbound)vertex.

edge-collection.inEdges(vertices)

Theedgesoperatorfindsalledgesendingin(inbound)adocumentfromvertices,whichmustalistofdocumentsordocumenthandles.

Examples

........>{label:"knows"});

arangosh>db.relation.inEdges(myGraph.v1._id);

arangosh>db.relation.inEdges(myGraph.v2._id);

showexecutionresultsedge-collection.outEdges(vertex)

Theedgesoperatorfindsalledgesstartingfrom(outbound)vertices.

edge-collection.outEdges(vertices)

Theedgesoperatorfindsalledgesstartingfrom(outbound)adocumentfromvertices,whichmustalistofdocumentsordocumenthandles.

Examples

CollectionMethods

........>{label:"knows"});

arangosh>db.relation.outEdges(myGraph.v1._id);

arangosh>db.relation.outEdges(myGraph.v2._id);

collection.iterate(iterator,options)

Iteratesoversomeelementsofthecollectionandapplythefunctioniteratortotheelements.Thefunctionwillbecalledwiththedocumentasfirstargumentandthecurrentnumber(startingwith0)assecondargument.

optionsmustbeanobjectwiththefollowingattributes:

limit(optional,defaultnone):useatmostlimitdocuments.

probability(optional,defaultall):anumberbetween0and1.Documentsarechosenwiththisprobability.

Examples

arangosh>for(i=-90;i<=90;i+=10){

........>for(j=-180;j<=180;j+=10){

........>db.example.save({name:"Name/"+i+"/"+j,

........>home:[i,j],

........>work:[-i,-j]});

........>}

........>

arangosh>db.example.ensureIndex({type:"geo",fields:["home"]});

arangosh>items=db.example.getIndexes().map(function(x){returnx.id;});

........>db.example.index(items[1]);

CollectionMethods

DatabaseMethods

Document

db._document(object)

The_documentmethodfindsadocumentgivenanobjectobjectcontainingthe_idattribute.Themethodreturnsthedocumentifitcanbefound.

Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.Anerrorisalsothrownifnodocumentexistswiththegiven_id.

Pleasenotethatifthemethodisexecutedonthearangodserver(e.g.frominsideaFoxxapplication),animmutabledocumentobjectwillbereturnedforperformancereasons.Itisnotpossibletochangeattributesofthisimmutableobject.Toupdateorpatchthereturneddocument,itneedstobecloned/copiedintoaregularJavaScriptobjectfirst.Thisisnotnecessaryifthe_documentmethodiscalledfromoutofarangoshorfromanyotherclient.

db._document(document-handle)

Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.Norevisioncanbespecifiedinthiscase.

Examples

Returnsthedocument:

arangosh>db._document("example/12345");

Existsdb._exists(object)

The_existsmethoddetermineswhetheradocumentexistsgivenanobjectobjectcontainingthe_idattribute.

Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.

Insteadofreturningthefounddocumentoranerror,thismethodwillonlyreturnanobjectwiththeattributes_id,_keyand_rev,orfalseifnodocumentwiththegiven_idor_keyexists.Itcanthusbeusedforeasyexistencechecks.

Thismethodwillthrowanerrorifusedimproperly,e.g.whencalledwithanon-documenthandle,anon-document,orwhenacross-collectionrequestisperformed.

db._exists(document-handle)

Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.

Inthecaseofarevisionmismatch_existsnowthrowsanerrorinsteadofsimplyreturningfalse.Thisistomakeitpossibletotellthedifferencebetweenarevisionmismatchandanon-existingdocument.

Replace

db._replace(selector,data)

Replacesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocumentwiththat_idinthecurrentdatabase.Thisdocumentisthenreplacedwiththedatagivenassecondargument.Anyattribute_id,_keyor_revindataisignored.

Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleoftheupdateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainstherevisionoftheold(nowreplaced)document.

DatabaseMethods

collection.replace(selector,data,options)

waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.

db._replace(document-handle,data)

db._replace(document-handle,data,options)

Examples

Createandreplaceadocument:

arangosh>a2=db._replace(a1,{a:2});

Theoptionssilent,returnNewandreturnOldarenew.

Update

db._update(selector,data)

Updatesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocumentwiththat_idinthecurrentdatabase.Thisdocumentisthenpatchedwiththedatagivenassecondargument.Anyattribute_id,_keyor_revindataisignored.

Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleoftheupdateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainstherevisionoftheold(nowupdated)document.

db._update(selector,data,options)

waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.

DatabaseMethods

silent:Ifthisflagissettotrue,nooutputisreturned.keepNull:TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesarestoredinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnullvalueswillberemovedfromthetargetdocument.mergeObjects:Controlswhetherobjects(notarrays)willbemergedifpresentinboththeexistingandthepatchdocument.Ifsettofalse,thevalueinthepatchdocumentwilloverwritetheexistingdocument'svalue.Ifsettotrue,objectswillbemerged.Thedefaultistrue.

db._update(document-handle,data)

db._update(document-handle,data,options)

Examples

arangosh>a2=db._update(a1,{b:2});

arangosh>a3=db._update(a1,{c:3});

Theoptionssilent,returnNewandreturnOldarenew.

Removedb._remove(selector)

Removesadocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocumentwiththat_idinthecurrentdatabase.Thisdocumentisthenremoved.

Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleoftheremoveddocument,theattribute_revcontainsthedocumentrevisionoftheremovedeocument.

db._remove(selector,options)

waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.

db._remove(document-handle)

db._remove(document-handle,options)

Examples

Removeadocument:

DatabaseMethods

arangosh>db._remove(a1);

arangosh>db._remove(a1,{overwrite:true});

showexecutionresultsRemovethedocumentintherevisiona1withaconflict:

arangosh>db._remove(a1);

arangosh>db._remove(a1,{overwrite:true});

arangosh>db._document(a1);

showexecutionresultsRemoveadocumentusingnewsignature:

arangosh>db.example.insert({_key:"11265325374",a:1});

arangosh>db.example.remove("example/11265325374",

........>{overwrite:true,waitForSync:false})

Themethodnowreturnsnotonlytruebutinformationabouttheremoveddocument(s).TheoptionssilentandreturnOldarenew.

DatabaseMethods

Graphs,Vertices&EdgesGraphs,vertices&edgesaredefinedintheGraphschapterindetails.

Relatedblogposts:

Graphsindatamodeling-istheemperornaked?IndexFreeAdjacencyorHybridIndexesforGraphDatabases

Graphs,Vertices&Edges

JavaScriptInterfacetoViewsThisisanintroductiontoArangoDB'sinterfaceforviewsandhowtohandleviewsfrontheJavaScriptshellarangosh.ForotherlanguagesseethecorrespondinglanguageAPI.

AddressofaView

AllviewsinArangoDBhaveauniqueidentifierandauniquename.ArangoDBinternallyusestheview'suniqueidentifiertolookupviews.Thisidentifier,however,ismanagedbyArangoDBandtheuserhasnocontroloverit.Inordertoallowuserstousetheirownnames,eachviewalsohasauniquenamewhichisspecifiedbytheuser.Toaccessaviewfromtheuserperspective,theviewnameshouldbeused,i.e.:

db._view(view-name)

Aviewiscreatedbya"db._createView"call.Thereturnedobjectmaythenbeusedviatheexposedmethods.

Forexample:Assumethattheviewidentifieris7254820andthenameisdemo,thentheviewcanbeaccessedas:

db._view("demo")

Ifnoviewwithsuchanameexists,thennullisreturned.

Createdb._createView(view-name,view-type,view-properties)

Thiscallwillcreateanewviewcalledview-name.ThismethodisadatabasemethodandisdocumentedindetailinDatabaseMethods

ViewTypes

Thecurrentlysupportedviewimplementationis:arangosearchasdescribedinArangoSearchView.

DatabaseMethods

db._view(view-name)

Returnstheviewwiththegivennameornullifnosuchviewexists.

db._view(view-identifier)

Returnstheviewwiththegivenidentifierornullifnosuchviewexists.Accessingviewsbyidentifierisdiscouragedforendusers.Endusersshouldaccessviewsusingtheviewname.

Examples

Getaviewbyname:

arangosh>db._view("demo");

Getaviewbyid:

arangosh>db._view(123456);

[ArangoView123456,"demo"]

Unknownview:

arangosh>db._view("unknown");

Createdb._createView(view-name,view-type,view-properties)

view-typemustbeoneofthesupportedViewTypes

view-propertiesviewconfigurationspecifictoeachview-type

Createsanewviewnamedview-nameoftypeview-typewithpropertiesview-properties.Iftheviewnamealreadyexistsorifthenameformatisinvalid,anerroristhrown.Formoreinformationonvalidviewnamespleaserefertothenamingconventions.

Examples

Createaview:

arangosh>v=db._createView("example",\<view-type\>,\<view-properties\>);

arangosh>v.properties();

arangosh>db._dropView("example");

AllViewsdb._views()

Returnsallviewsofthegivendatabase.

Examples

Queryviews:

arangosh>db._createView("example",\<view-type\>,\<view-properties\>);

arangosh>db._views();

DatabaseMethods

db._dropView(view-name)

Dropsaviewnamedview-nameandallitsdata.Noerroristhrownifthereisnosuchview.

db._dropView(view-identifier)

Dropsaviewidentifiedbyview-identifierwithallitsdata.Noerroristhrownifthereisnosuchview.

Examples

Dropaview:

arangosh>v=db._view("example");

arangosh>v;

DatabaseMethods

ViewMethods

view.drop()

Dropsaviewandallitsdata.

Examples

Dropaview:

arangosh>v.drop();

arangosh>v;

QueryName

view.name()

Returnsthenameoftheview.

Examples

Getviewname:

arangosh>v.name();

ModifyNameview.rename(new-name)

Renamesaviewusingthenew-name.Thenew-namemustnotalreadybeusedbyadifferentview.new-namemustalsobeavalidviewname.Formoreinformationonvalidviewnamespleaserefertothenamingconventions.

Ifrenamingfailsforanyreason,anerroristhrown.

Examples

arangosh>v.name();

arangosh>v.rename("example-renamed");

arangosh>v.name();

arangosh>db._dropView("example-renamed");

QueryType

view.type()

Returnsthetypeoftheview.

Examples

Getviewtype:

arangosh>v.type();

ViewMethods

QueryProperties

view.properties()

Returnsthepropertiesoftheview.TheformatoftheresultisspecifictoeachofthesupportedViewTypes.

Examples

Getviewproperties:

arangosh>v.properties();

ModifyPropertiesview.properties(view-property-modification)

Modifiesthepropertiesoftheview.TheformatoftheresultisspecifictoeachofthesupportedViewTypes.

Examples

Modifyviewproperties:

arangosh>v.properties(\<view-property-modification\>);

ViewMethods

NamingConventionsinArangoDBThefollowingnamingconventionsshouldbefollowedbyuserswhencreatingdatabases,collectionsanddocumentsinArangoDB.

NamingConventions

DatabaseNamesArangoDBwillalwaysstartupwithadefaultdatabase,named_system.UserscancreateadditionaldatabasesinArangoDB,providedthedatabasenamesconformtothefollowingconstraints:

Databasenamesmustonlyconsistofthelettersatoz(bothloweranduppercaseallowed),thenumbers0to9,andtheunderscore(_)ordash(-)symbolsThisalsomeansthatanynon-ASCIIdatabasenamesarenotallowedDatabasenamesmustalwaysstartwithaletter.Databasenamesstartingwithanunderscoreareconsideredtobesystemdatabases,andusersshouldnotcreateordeletethoseThemaximumallowedlengthofadatabasenameis64bytesDatabasenamesarecase-sensitive

DatabaseNames

CollectionNamesUserscanpicknamesfortheircollectionsasdesired,providedthefollowingnamingconstraintsarenotviolated:

Collectionnamesmustonlyconsistofthelettersatoz(bothinloweranduppercase),thenumbers0to9,andtheunderscore(_)ordash(-)symbols.Thisalsomeansthatanynon-ASCIIcollectionnamesarenotallowedUser-definedcollectionnamesmustalwaysstartwithaletter.Systemcollectionnamesmuststartwithanunderscore.AllcollectionnamesstartingwithanunderscoreareconsideredtobesystemcollectionsthatareforArangoDB'sinternaluseonly.SystemcollectionnamesshouldnotbeusedbyendusersfortheirowncollectionsThemaximumallowedlengthofacollectionnameis64bytesCollectionnamesarecase-sensitive

CollectionNames

DocumentKeysUserscandefinetheirownkeysfordocumentstheysave.Thedocumentkeywillbesavedalongwithadocumentinthe_keyattribute.Userscanpickkeyvaluesasrequired,providedthatthevaluesconformtothefollowingrestrictions:

Thekeymustbeastringvalue.Numerickeysarenotallowed,butanynumericvaluecanbeputintoastringandcanthenbeusedasdocumentkey.Thekeymustbeatleast1byteandatmost254byteslong.Emptykeysaredisallowedwhenspecified(thoughitmaybevalidtocompletelyomitthe_keyattributefromadocument)Itmustconsistofthelettersa-z(loweroruppercase),thedigits0-9oranyofthefollowingpunctuationcharacters:_-:.@()+,=;$!*'%Anyothercharacters,especiallymulti-byteUTF-8sequences,whitespaceorpunctuationcharacterscannotbeusedinsidekeyvaluesThekeymustbeuniquewithinthecollectionitisused

Keysarecase-sensitive,i.e.myKeyandMyKEYareconsideredtobedifferentkeys.

Specifyingadocumentkeyisoptionalwhencreatingnewdocuments.Ifnodocumentkeyisspecifiedbytheuser,ArangoDBwillcreatethedocumentkeyitselfaseachdocumentisrequiredtohaveakey.

Therearenoguaranteesabouttheformatandpatternofauto-generateddocumentkeysotherthantheaboverestrictions.Clientsshouldthereforetreatauto-generateddocumentkeysasopaquevaluesandnotrelyontheirformat.

Thecurrentformatforgeneratedkeysisastringcontainingnumericdigits.Thenumericvaluesreflectchronologicaltimeinthesensethat_keyvaluesgeneratedlaterwillcontainhighernumbersthan_keyvaluesgeneratedearlier.Buttheexactvaluethatwillbegeneratedbytheserverisnotpredictable.Notethatifyousortonthe_keyattribute,stringcomparisonwillbeused,whichmeans"100"islessthan"99"etc.

DocumentKeys

AttributeNamesUserscanpickattributenamesfordocumentattributesasdesired,providedthefollowingattributenamingconstraintsarenotviolated:

AttributenamesstartingwithanunderscoreareconsideredtobesystemattributesforArangoDB'sinternaluse.SuchattributenamesarealreadyusedbyArangoDBforspecialpurposes:

_idisusedtocontainadocument'shandle_keyisusedtocontainadocument'suser-definedkey_revisusedtocontainthedocument'srevisionnumberInedgecollections,the

_from_to

attributesareusedtoreferenceotherdocuments.

Moresystemattributesmaybeaddedinthefuturewithoutfurthernoticesoendusersshouldtrytoavoidusingtheirownattributenamesstartingwithunderscores.

Theoretically,attributenamescanincludepunctuationandspecialcharactersasdesired,providedthenameisavalidUTF-8string.Formaximumportability,specialcharactersshouldbeavoidedthough.Forexample,attributenamesmaycontainthedotsymbol,butthedothasaspecialmeaninginJavaScriptandalsoinAQL,sowhenusingsuchattributenamesinoneoftheselanguages,theattributenameneedstobequotedbytheenduser.Overallitmightbebettertouseattributenameswhichdon'trequireanyquoting/escapinginalllanguagesused.Thisincludeslanguagesusedbytheclient(e.g.Ruby,PHP)iftheattributesaremappedtoobjectmembersthere.

Attributenamesstartingwithanat-mark(@)willneedtobeenclosedinbacktickswhenusedinanAQLquerytotellthemapartfrombindvariables.Thereforewedonotencouragetheuseofattributesstartingwithat-marks,thoughtheywillworkwhenusedproperly.ArangoDBdoesnotenforcealengthlimitforattributenames.However,longattributenamesmayusemorememoryinresultsetsetc.Thereforetheuseoflongattributenamesisdiscouraged.Attributenamesarecase-sensitive.Attributeswithemptynames(anemptystring)aredisallowed.

AttributeNames

HandlingIndexesThisisanintroductiontoArangoDB'sinterfaceforindexesingeneral.Therearespecialsectionsfor

IndexBasics:IntroductiontoallindextypesWhichindextousewhen:IndextypeandoptionsadviserIndexUtilization:HowArangoDBusesindexesWorkingwithIndexes:Howtohandleindexesprogrammaticallyusingthedbobject

HashIndexesSkiplistsPersistentIndexesFulltextIndexesGeo-spatialIndexesVertex-centricIndexes

Indexing

Indexbasics

Indexesallowfastaccesstodocuments,providedtheindexedattribute(s)areusedinaquery.WhileArangoDBautomaticallyindexessomesystemattributes,usersarefreetocreateextraindexesonnon-systemattributesofdocuments.

User-definedindexescanbecreatedoncollectionlevel.Mostuser-definedindexescanbecreatedbyspecifyingthenamesoftheindexattributes.Someindextypesallowindexingjustoneattribute(e.g.fulltextindex)whereasotherindextypesallowindexingmultipleattributesatthesametime.

Thesystemattributes_id,_key,_fromand_toareautomaticallyindexedbyArangoDB,withouttheuserbeingrequiredtocreateextraindexesforthem._idand_keyarecoveredbyacollection'sprimarykey,and_fromand_toarecoveredbyanedgecollection'sedgeindexautomatically.

Usingthesystemattribute_idinuser-definedindexesisnotpossible,butindexing_key,_rev,_from,and_tois.

ArangoDBprovidesthefollowingindextypes:

PrimaryIndex

Foreachcollectiontherewillalwaysbeaprimaryindexwhichisahashindexforthedocumentkeys(_keyattribute)ofalldocumentsinthecollection.Theprimaryindexallowsquickselectionofdocumentsinthecollectionusingeitherthe_keyor_idattributes.ItwillbeusedfromwithinAQLqueriesautomaticallywhenperformingequalitylookupson_keyor_id.

Therearealsodedicatedfunctionstofindadocumentgivenits_keyor_idthatwillalwaysmakeuseoftheprimaryindex:

db.collection.document("<document-key>");

db._document("<document-id>");

Astheprimaryindexisanunsortedhashindex,itcannotbeusedfornon-equalityrangequeriesorforsorting.

Theprimaryindexofacollectioncannotbedroppedorchanged,andthereisnomechanismtocreateuser-definedprimaryindexes.

EdgeIndex

Everyedgecollectionalsohasanautomaticallycreatededgeindex.Theedgeindexprovidesquickaccesstodocumentsbyeithertheir_fromor_toattributes.Itcanthereforebeusedtoquicklyfindconnectionsbetweenvertexdocumentsandisinvokedwhentheconnectingedgesofavertexarequeried.

EdgeindexesareusedfromwithinAQLwhenperformingequalitylookupson_fromor_tovaluesinanedgecollections.Therearealsodedicatedfunctionstofindedgesgiventheir_fromor_tovaluesthatwillalwaysmakeuseoftheedgeindex:

db.collection.edges("<from-value>");

db.collection.edges("<to-value>");

db.collection.outEdges("<from-value>");

db.collection.outEdges("<to-value>");

db.collection.inEdges("<from-value>");

db.collection.inEdges("<to-value>");

Internally,theedgeindexisimplementedasahashindex,whichstorestheunionofall_fromand_toattributes.Itcanbeusedforequalitylookups,butnotforrangequeriesorforsorting.Edgeindexesareautomaticallycreatedforedgecollections.Itisnotpossibletocreateuser-definededgeindexes.However,itispossibletofreelyusethe_fromand_toattributesinuser-definedindexes.

Anedgeindexcannotbedroppedorchanged.

HashIndex

Ahashindexcanbeusedtoquicklyfinddocumentswithspecificattributevalues.Thehashindexisunsorted,soitsupportsequalitylookupsbutnorangequeriesorsorting.

IndexBasics

Ahashindexcanbecreatedononeormultipledocumentattributes.Ahashindexwillonlybeusedbyaqueryifallindexattributesarepresentinthesearchcondition,andifallattributesarecomparedusingtheequality(==)operator.HashindexesareusedfromwithinAQLandseveralqueryfunctions,e.g.byExample,firstExampleetc.

Hashindexescanoptionallybedeclaredunique,thendisallowingsavingthesamevalue(s)intheindexedattribute(s).Hashindexescanoptionallybesparse.

Thedifferenttypesofhashindexeshavethefollowingcharacteristics:

uniquehashindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.Tryingtoinsertadocumentwiththesamekeyvalueasanalreadyexistingdocumentwillleadtoauniqueconstraintviolation.

Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueofnullintheindexattribute(s)willstillbeindexed.Akeyvalueofnullmayonlyoccuronceintheindex,sothistypeofindexcannotbeusedforoptionalattributes.

Theuniqueoptioncanalsobeusedtoensurethatnoduplicateedgesarecreated,byaddingacombinedindexforthefields_fromand_totoanedgecollection.

unique,sparsehashindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.Documentsinwhichatleastoneoftheindexattributesisnotsetorhasavalueofnullarenotincludedintheindex.Thistypeofindexcanbeusedtoensurethattherearenoduplicatekeysinthecollectionfordocumentswhichhavetheindexedattributesset.Astheindexwillexcludedocumentsforwhichtheindexedattributesarenullornotset,itcanbeusedforoptionalattributes.

non-uniquehashindex:alldocumentsinthecollectionwillbeindexed.Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueofnullintheindexattribute(s)willstillbeindexed.Duplicatekeyvaluescanoccuranddonotleadtouniqueconstraintviolations.

non-unique,sparsehashindex:onlythosedocumentswillbeindexedthathavealltheindexedattributessettoavalueotherthannull.Itcanbeusedforoptionalattributes.

Theamortizedcomplexityoflookup,insert,update,andremovaloperationsinuniquehashindexesisO(1).

Non-uniquehashindexeshaveanamortizedcomplexityofO(1)forinsert,update,andremovaloperations.Thatmeansnon-uniquehashindexescanbeusedonattributeswithlowcardinality.

Ifahashindexiscreatedonanattributethatismissinginallormanyofthedocuments,thebehaviorisasfollows:

iftheindexissparse,thedocumentsmissingtheattributewillnotbeindexedandnotuseindexmemory.Thesedocumentswillnotinfluencetheupdateorremovalperformancefortheindex.

iftheindexisnon-sparse,thedocumentsmissingtheattributewillbecontainedintheindexwithakeyvalueofnull.

Hashindexessupportindexingarrayvaluesiftheindexattributenameisextendedwitha[*].

SkiplistIndex

Askiplistisasortedindexstructure.Itcanbeusedtoquicklyfinddocumentswithspecificattributevalues,forrangequeriesandforreturningdocumentsfromtheindexinsortedorder.SkiplistswillbeusedfromwithinAQLandseveralqueryfunctions,e.g.byExample,firstExampleetc.

Skiplistindexeswillbeusedforlookups,rangequeriesandsortingonlyifeitherallindexattributesareprovidedinaquery,orifaleftmostprefixoftheindexattributesisspecified.

Forexample,ifaskiplistindexiscreatedonattributesvalue1andvalue2,thefollowingfilterconditionscanusetheindex(note:the<=and>=operatorsareintentionallyomittedhereforthesakeofbrevity):

FILTERdoc.value1==...

FILTERdoc.value1<...

FILTERdoc.value1>...

FILTERdoc.value1>...&&doc.value1<...

FILTERdoc.value1==...&&doc.value2==...

FILTERdoc.value1==...&&doc.value2>...

FILTERdoc.value1==...&&doc.value2>...&&doc.value2<...

IndexBasics

Inordertouseaskiplistindexforsorting,theindexattributesmustbespecifiedintheSORTclauseofthequeryinthesameorderastheyappearintheindexdefinition.Skiplistindexesarealwayscreatedinascendingorder,buttheycanbeusedtoaccesstheindexedelementsinbothascendingordescendingorder.However,foracombinedindex(anindexonmultipleattributes)thisrequiresthatthesortordersinasinglequeryasspecifiedintheSORTclausemustbeeitherallascending(optionallyommittedasascendingisthedefault)oralldescending.

Forexample,iftheskiplistindexiscreatedonattributesvalue1andvalue2(inthisorder),thenthefollowingsortsclausescanusetheindexforsorting:

SORTvalue1ASC,value2ASC(anditsequivalentSORTvalue1,value2)SORTvalue1DESC,value2DESC

SORTvalue1ASC(anditsequivalentSORTvalue1)SORTvalue1DESC

Thefollowingsortclausescannotmakeuseoftheindexorder,andrequireanextrasortstep:

SORTvalue1ASC,value2DESC

SORTvalue1DESC,value2ASC

SORTvalue2(anditsequivalentSORTvalue2ASC)SORTvalue2DESC(becausefirstindexedattributevalue1isnotusedinsortclause)

Note:thelattertwosortclausescannotusetheindexbecausethesortclausedoesnotrefertoaleftmostprefixoftheindexattributes.

Skiplistscanoptionallybedeclaredunique,disallowingsavingthesamevalueintheindexedattribute.Theycanbesparseornon-sparse.

Thedifferenttypesofskiplistindexeshavethefollowingcharacteristics:

uniqueskiplistindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.Tryingtoinsertadocumentwiththesamekeyvalueasanalreadyexistingdocumentwillleadtoauniqueconstraintviolation.

Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueofnullintheindexattribute(s)willstillbeindexed.Akeyvalueofnullmayonlyoccuronceintheindex,sothistypeofindexcannotbeusedforoptionalattributes.

unique,sparseskiplistindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.Documentsinwhichatleastoneoftheindexattributesisnotsetorhasavalueofnullarenotincludedintheindex.Thistypeofindexcanbeusedtoensurethattherearenoduplicatekeysinthecollectionfordocumentswhichhavetheindexedattributesset.Astheindexwillexcludedocumentsforwhichtheindexedattributesarenullornotset,itcanbeusedforoptionalattributes.

non-uniqueskiplistindex:alldocumentsinthecollectionwillbeindexed.Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueofnullintheindexattribute(s)willstillbeindexed.Duplicatekeyvaluescanoccuranddonotleadtouniqueconstraintviolations.

non-unique,sparseskiplistindex:onlythosedocumentswillbeindexedthathavealltheindexedattributessettoavalueotherthannull.Itcanbeusedforoptionalattributes.

Theoperationalamortizedcomplexityforskiplistindexesislogarithmicallycorrelatedwiththenumberofdocumentsintheindex.

Skiplistindexessupportindexingarrayvaluesiftheindexattributenameisextendedwitha[*]`.

PersistentIndex

Thepersistentindexisasortedindexwithpersistence.Theindexentriesarewrittentodiskwhendocumentsarestoredorupdated.Thatmeanstheindexentriesdonotneedtoberebuiltfromthecollectiondatawhentheserverisrestartedortheindexedcollectionisinitiallyloaded.Thususingpersistentindexesmayreducecollectionloadingtimes.

Thepersistentindextypecanbeusedforsecondaryindexesatthemoment.Thatmeansthepersistentindexcurrentlycannotbemadetheonlyindexforacollection,becausetherewillalwaysbethein-memoryprimaryindexforthecollectioninaddition,andpotentiallymoreindexes(suchastheedgesindexforanedgecollection).

TheindeximplementationisusingtheRocksDBengine,anditprovideslogarithmiccomplexityforinsert,update,andremoveoperations.Asthepersistentindexisnotanin-memoryindex,itdoesnotstorepointersintotheprimaryindexasallthein-memoryindexesdo,butinsteaditstoresadocument'sprimarykey.Toretrieveadocumentviaapersistentindexviaanindexvaluelookup,therewillthereforebe

IndexBasics

anadditionalO(1)lookupintotheprimaryindextofetchtheactualdocument.

Asthepersistentindexissorted,itcanbeusedforpointlookups,rangequeriesandsortingoperations,butonlyifeitherallindexattributesareprovidedinaquery,orifaleftmostprefixoftheindexattributesisspecified.

GeoIndex

Userscancreateadditionalgeoindexesononeormultipleattributesincollections.Ageoindexisusedtofindplacesonthesurfaceoftheearthfast.

Thegeoindexstorestwo-dimensionalcoordinates.Itcanbecreatedoneithertwoseparatedocumentattributes(latitudeandlongitude)orasinglearrayattributethatcontainsbothlatitudeandlongitude.Latitudeandlongitudemustbenumericvalues.

Thegeoindexprovidesoperationstofinddocumentswithcoordinatesnearesttoagivencomparisoncoordinate,andtofinddocumentswithcoordinatesthatarewithinaspecifiableradiusaroundacomparisoncoordinate.

ThegeoindexisusedviadedicatedfunctionsinAQL,thesimplequeriesfunctionsanditisimplicitlyappliedwheninAQLaSORTorFILTERisusedwiththedistancefunction.Otherwiseitwillnotbeusedforothertypesofqueriesorconditions.

FulltextIndex

Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.Afulltextindexcanbecreatedonasingleattributeonly,andwillindexallwordscontainedindocumentsthathaveatextualvalueinthatattribute.Onlywordswitha(specifiable)minimumlengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedbylibicu,whichistakingintoaccounttheselectedlanguageprovidedatserverstart.Wordsareindexedintheirlower-casedform.Theindexsupportscompletematchqueries(fullwords)andprefixqueries,plusbasiclogicaloperationssuchasand,orandnotforcombiningpartialresults.

Thefulltextindexissparse,meaningitwillonlyindexdocumentsforwhichtheindexattributeissetandcontainsastringvalue.Additionally,onlywordswithaconfigurableminimumlengthwillbeincludedintheindex.

ThefulltextindexisusedviadedicatedfunctionsinAQLorthesimplequeries,butwillnotbeenabledforothertypesofqueriesorconditions.

Indexingattributesandsub-attributes

Top-levelaswellasnestedattributescanbeindexed.Forattributesatthetoplevel,theattributenamesalonearerequired.Toindexasinglefield,passanarraywithasingleelement(stringoftheattributekey)tothefieldsparameteroftheensureIndex()method.Tocreateacombinedindexovermultiplefields,simplyaddmorememberstothefieldsarray:

//{name:"Smith",age:35}

db.posts.ensureIndex({type:"hash",fields:["name"]})

db.posts.ensureIndex({type:"hash",fields:["name","age"]})

Toindexsub-attributes,specifytheattributepathusingthedotnotation:

//{name:{last:"Smith",first:"John"}}

db.posts.ensureIndex({type:"hash",fields:["name.last"]})

db.posts.ensureIndex({type:"hash",fields:["name.last","name.first"]})

Indexingarrayvalues

Ifanindexattributecontainsanarray,ArangoDBwillstoretheentirearrayastheindexvaluebydefault.Accessingindividualmembersofthearrayviatheindexisnotpossiblethisway.

Tomakeanindexinserttheindividualarraymembersintotheindexinsteadoftheentirearrayvalue,aspecialarrayindexneedstobecreatedfortheattribute.Arrayindexescanbesetuplikeregularhashorskiplistindexesusingthecollection.ensureIndex()function.Tomakeahashorskiplistindexanarrayindex,theindexattributenameneedstobeextendedwith[*]whencreatingtheindexandwhenfilteringinanAQLqueryusingtheINoperator.

Thefollowingexamplecreatesanarrayhashindexonthetagsattributeinacollectionnamedposts:

IndexBasics

db.posts.ensureIndex({type:"hash",fields:["tags[*]"]});

db.posts.insert({tags:["foobar","baz","quux"]});

ThisarrayindexcanthenbeusedforlookingupindividualtagsvaluesfromAQLqueriesviatheINoperator:

FORdocINposts

FILTER'foobar'INdoc.tags

RETURNdoc

Itispossibletoaddthearrayexpansionoperator[*],butitisnotmandatory.Youmayuseittoindicatethatanarrayindexisused,itispurelycosmetichowever:

FORdocINposts

FILTER'foobar'INdoc.tags[*]

RETURNdoc

ThefollowingFILTERconditionswillnotusethearrayindex:

FILTERdoc.tagsANY=='foobar'

FILTERdoc.tagsANYIN'foobar'

FILTERdoc.tagsIN'foobar'

FILTERdoc.tags=='foobar'

FILTER'foobar'==doc.tags

Itisalsopossibletocreateanindexonsubattributesofarrayvalues.Thismakessenseiftheindexattributeisanarrayofobjects,e.g.

db.posts.ensureIndex({type:"hash",fields:["tags[*].name"]});

db.posts.insert({tags:[{name:"foobar"},{name:"baz"},{name:"quux"}]});

Thefollowingquerywillthenusethearrayindex(thisdoesrequirethearrayexpansionoperator):

FORdocINposts

FILTER'foobar'INdoc.tags[*].name

RETURNdoc

Ifyoustoreadocumenthavingthearraywhichdoescontainelementsnothavingthesubattributesthisdocumentwillalsobeindexedwiththevaluenull,whichinArangoDBisequaltoattributenotexisting.

ArangoDBsupportscreatingarrayindexeswithasingle[*]operatorperindexattribute.Forexample,creatinganindexasfollowsisnotsupported:

db.posts.ensureIndex({type:"hash",fields:["tags[*].name[*].value"]});

Arrayvalueswillautomaticallybede-duplicatedbeforebeinginsertedintoanarrayindex.Forexample,ifthefollowingdocumentisinsertedintothecollection,theduplicatearrayvaluebarwillbeinsertedonlyonce:

db.posts.insert({tags:["foobar","bar","bar"]});

Thisisdonetoavoidredudantstorageofthesameindexvalueforthesamedocument,whichwouldnotprovideanybenefit.

Ifanarrayindexisdeclaredunique,thede-duplicationofarrayvalueswillhappenbeforeinsertingthevaluesintotheindex,sotheaboveinsertoperationwithtwoidenticalvaluesbarwillnotnecessarilyfail

Itwillalwaysfailiftheindexalreadycontainsaninstanceofthebarvalue.However,ifthevaluebarisnotalreadypresentintheindex,thenthede-duplicationofthearrayvalueswilleffectivelyleadtobarbeinginsertedonlyonce.

Toturnoffthededuplicationofarrayvalues,itispossibletosetthededuplicateattributeonthearrayindextofalse.Thedefaultvaluefordeduplicateistruehowever,sode-duplicationwilltakeplaceifnotexplicitlyturnedoff.

db.posts.ensureIndex({type:"hash",fields:["tags[*]"],deduplicate:false});

IndexBasics

//willfailnow

db.posts.insert({tags:["foobar","bar","bar"]});

Ifanarrayindexisdeclaredandyoustoredocumentsthatdonothaveanarrayatthespecifiedattributethisdocumentwillnotbeinsertedintheindex.Hencethefollowingobjectswillnotbeindexed:

db.posts.insert({something:"else"});

db.posts.insert({tags:null});

db.posts.insert({tags:"thisisnoarray"});

db.posts.insert({tags:{content:[1,2,3]}});

Anarrayindexisabletoindexexplicitnullvalues.Whenqueriedfornullvalues,itwillonlyreturnthosedocumentshavingexplicitlynullstoredinthearray,itwillnotreturnanydocumentsthatdonothavethearrayatall.

db.posts.insert({tags:null})//Willnotbeindexed

db.posts.insert({tags:[]})//Willnotbeindexed

db.posts.insert({tags:[null]});//Willbeindexedfornull

db.posts.insert({tags:[null,1,2]});//Willbeindexedfornull,1and2

Declaringanarrayindexassparsedoesnothaveaneffectonthearraypartoftheindex,thisinparticularmeansthatexplicitnullvaluesarealsoindexedinthesparseversion.Ifanindexiscombinedfromanarrayandanormalattributethesparsitywillapplyfortheattributee.g.:

db.posts.ensureIndex({type:"hash",fields:["tags[*]","name"],sparse:true});

db.posts.insert({tags:null,name:"alice"})//Willnotbeindexed

db.posts.insert({tags:[],name:"alice"})//Willnotbeindexed

db.posts.insert({tags:[1,2,3]})//Willnotbeindexed

db.posts.insert({tags:[1,2,3],name:null})//Willnotbeindexed

db.posts.insert({tags:[1,2,3],name:"alice"})

//Willbeindexedfor[1,"alice"],[2,"alice"],[3,"alice"]

db.posts.insert({tags:[null],name:"bob"})

//Willbeindexedfor[null,"bob"]

PleasenotethatfilteringusingarrayindexesonlyworksfromwithinAQLqueriesandonlyifthequeryfiltersontheindexedattributeusingtheINoperator.Theothercomparisonoperators(==,!=,>,>=,<,<=,ANY,ALL,NONE)currentlycannotusearrayindexes.

Vertexcentricindexes

Asmentionedabove,themostimportantindexesforgraphsaretheedgeindexes,indexingthe_fromand_toattributesofedgecollections.Theyprovideveryquickaccesstoalledgesoriginatinginorarrivingatagivenvertex,whichallowstoquicklyfindallneighboursofavertexinagraph.

Inmanycasesonewouldliketorunmorespecificqueries,forexamplefindingamongsttheedgesoriginatinginagivenvertexonlythosewiththe20latesttimestamps.Exactlythisisachievedwith"vertexcentricindexes".Inasensethesearelocalizedindexesforanedgecollection,whichsitateverysinglevertex.

Technically,theyareimplementedinArangoDBasindexes,whichsortthecompleteedgecollectionfirstby_fromandthenbyotherattributes.Ifweforexamplehaveaskiplistindexontheattributes_fromandtimestampofanedgecollection,wecananswertheabovequestionveryquicklywithasinglerangelookupintheindex.

SinceArangoDB3.0onecancreatesortedindexes(type"skiplist"and"persistent")thatindexthespecialedgeattributes_fromor_toandadditionallyotherattributes.SinceArangoDB3.1,theseareusedingraphtraversals,whenappropriateFILTERstatementsarefoundbytheoptimizer.

Forexample,tocreateavertexcentricindexoftheabovetype,youwouldsimplydo

db.edges.ensureIndex({"type":"skiplist","fields":["_from","timestamp"]});

Then,querieslike

IndexBasics

FORv,e,pIN1..1OUTBOUND"V/1"edges

FILTERe.timestampALL>="2016-11-09"

RETURNp

willbeconsiderablyfasterincasetherearemanyedgesoriginatinginvertex"V/1"butonlyfewwitharecenttimestamp.

IndexBasics

WhichIndextousewhen

ArangoDBautomaticallyindexesthe_keyattributeineachcollection.Thereisnoneedtoindexthisattributeseparately.Pleasenotethatadocument's_idattributeisderivedfromthe_keyattribute,andisthusimplicitlyindexed,too.

ArangoDBwillalsoautomaticallycreateanindexon_fromand_toinanyedgecollection,meaningincomingandoutgoingconnectionscanbedeterminedefficiently.

Indextypes

Userscandefineadditionalindexesononeormultipledocumentattributes.SeveraldifferentindextypesareprovidedbyArangoDB.Theseindexeshavedifferentusagescenarios:

hashindex:providesquickaccesstoindividualdocumentsif(andonlyif)allindexedattributesareprovidedinthesearchquery.Theindexwillonlybeusedforequalitycomparisons.Itdoesnotsupportrangequeriesandcannotbeusedforsorting.

Thehashindexisagoodcandidateifallormostqueriesontheindexedattribute(s)areequalitycomparisons.TheuniquehashindexprovidesanamortizedcomplexityofO(1)forinsert,update,removeandlookupoperations.Thenon-uniquehashindexprovidesO(1)inserts,updatesandremoves,andwillallowlookingupdocumentsbyindexvaluewithamortizedO(n)complexity,withnbeingthenumberofdocumentswiththatindexvalue.

Anon-uniquehashindexonanoptionaldocumentattributeshouldbedeclaredsparsesothatitwillnotindexdocumentsforwhichtheindexattributeisnotset.

skiplistindex:skiplistskeeptheindexedvaluesinanorder,sotheycanbeusedforequalitylookups,rangequeriesandforsorting.Forhighselectivityattributes,skiplistindexeswillhaveahigheroverheadthanhashindexes.Forlowselectivityattributes,skiplistindexeswillbemoreefficientthannon-uniquehashindexes.

Additionally,skiplistindexesallowmoreusecases(e.g.rangequeries,sorting)thanhashindexes.Furthermore,theycanbeusedforlookupsbasedonaleftmostprefixoftheindexattributes.

persistentindex:apersistentindexbehavesmuchlikethesortedskiplistindex,exceptthatallindexvaluesarepersistedondiskanddonotneedtoberebuiltinmemorywhentheserverisrestartedortheindexedcollectionisreloaded.Theoperationsinapersistentindexhavelogarithmiccomplexity,butoperationshavemayhaveahigherconstantfactorthantheoperationsinaskiplistindex,becausethepersistentindexmayneedtomakeextraroundtripstotheprimaryindextofetchtheactualdocuments.

Apersistentindexcanbeusedforequalitylookups,rangequeriesandforsorting.Forhighselectivityattributes,persistentindexeswillhaveahigheroverheadthanskiplistorhashindexes.

Persistentindexesallowmoreusecases(e.g.rangequeries,sorting)thanhashindexes.Furthermore,theycanbeusedforlookupsbasedonaleftmostprefixoftheindexattributes.Incontrasttothein-memoryskiplistindexes,persistentindexesdonotneedtoberebuiltin-memorysotheydon'tinfluencetheloadingtimeofcollectionsasotherin-memoryindexesdo.

geoindex:thegeoindexprovidedbyArangoDBallowssearchingfordocumentswithinaradiusaroundatwo-dimensionalearthcoordinate(point),ortofinddocumentswithareclosesttoapoint.Documentcoordinatescaneitherbespecifiedintwodifferentdocumentattributesorinasingleattribute,e.g.

{"latitude":50.9406645,"longitude":6.9599115}

{"coords":[50.9406645,6.9599115]}

GeoindexeswillbeinvokedviaspecialfunctionsorAQLoptimization.TheoptimizationcanbetriggeredwhenacollectionwithgeoindexisenumeratedandaSORTorFILTERstatementisusedinconjunctionwiththedistancefunction.

fulltextindex:afulltextindexcanbeusedtoindexallwordscontainedinaspecificattributeofalldocumentsinacollection.Onlywordswitha(specifiable)minimumlengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedbylibicu,whichistakingintoaccounttheselectedlanguageprovidedatserverstart.

Whichindextousewhen

Theindexsupportscompletematchqueries(fullwords)andprefixqueries.Fulltextindexeswillonlybeinvokedviaspecialfunctions.

Sparsevs.non-sparseindexes

Hashindexesandskiplistindexescanoptionallybecreatedsparse.Asparseindexdoesnotcontaindocumentsforwhichatleastoneoftheindexattributeisnotsetorcontainsavalueofnull.

Assuchdocumentsareexcludedfromsparseindexes,theymaycontainfewerdocumentsthantheirnon-sparsecounterparts.Thisenablesfasterindexingandcanleadtoreducedmemoryusageincasetheindexedattributedoesoccuronlyinsome,butnotalldocumentsofthecollection.Sparseindexeswillalsoreducethenumberofcollisionsinnon-uniquehashindexesincasenon-existingoroptionalattributesareindexed.

Inordertocreateasparseindex,anobjectwiththeattributesparsecanbeaddedtotheindexcreationcommands:

db.collection.ensureIndex({type:"hash",fields:["attributeName"],sparse:true});

db.collection.ensureIndex({type:"hash",fields:["attributeName1","attributeName2"],sparse:true});

db.collection.ensureIndex({type:"hash",fields:["attributeName"],unique:true,sparse:true});

db.collection.ensureIndex({type:"hash",fields:["attributeName1","attributeName2"],unique:true,sparse:true});

db.collection.ensureIndex({type:"skiplist",fields:["attributeName"],sparse:true});

db.collection.ensureIndex({type:"skiplist",fields:["attributeName1","attributeName2"],sparse:true});

db.collection.ensureIndex({type:"skiplist",fields:["attributeName"],unique:true,sparse:true});

db.collection.ensureIndex({type:"skiplist",fields:["attributeName1","attributeName2"],unique:true,sparse:true});

Whennotexplicitlyset,thesparseattributedefaultstofalsefornewindexes.Otherindexesthanhashandskiplistdonotsupportsparsity.

Assparseindexesmayexcludesomedocumentsfromthecollection,theycannotbeusedforalltypesofqueries.Sparsehashindexescannotbeusedtofinddocumentsforwhichatleastoneoftheindexedattributeshasavalueofnull.Forexample,thefollowingAQLquerycannotuseasparseindex,evenifonewascreatedonattributeattr:

FORdocIncollection

FILTERdoc.attr==null

RETURNdoc

Ifthelookupvalueisnon-constant,asparseindexmayormaynotbeused,dependingontheothertypesofconditionsinthequery.Iftheoptimizercansafelydeterminethatthelookupvaluecannotbenull,asparseindexmaybeused.Whenuncertain,theoptimizerwillnotmakeuseofasparseindexinaqueryinordertoproducecorrectresults.

Forexample,thefollowingqueriescannotuseasparseindexonattrbecausetheoptimizerwillnotknowbeforehandwhetherthevalueswhicharecomparedtodoc.attrwillincludenull:

FORdocIncollection

FILTERdoc.attr==SOME_FUNCTION(...)

RETURNdoc

FORotherINotherCollection

FORdocIncollection

FILTERdoc.attr==other.attr

RETURNdoc

Sparseskiplistindexescanbeusedforsortingiftheoptimizercansafelydetectthattheindexrangedoesnotincludenullforanyoftheindexattributes.

Notethatifyouintendtousejoinsitmaybeclevertousenon-sparsityandmaybeevenuniquenessforthatattribute,elseallitemscontainingthenullvaluewillmatchagainsteachotherandthusproducelargeresults.

Whichindextousewhen

IndexUtilizationInmostcasesArangoDBwilluseasingleindexpercollectioninagivenquery.AQLqueriescanusemorethanoneindexpercollectionwhenmultipleFILTERconditionsarecombinedwithalogicalORandthesecanbecoveredbyindexes.AQLquerieswilluseasingleindexpercollectionwhenFILTERconditionsarecombinedwithlogicalAND.

Creatingmultipleindexesondifferentattributesofthesamecollectionmaygivethequeryoptimizermorechoiceswhenpickinganindex.Creatingmultipleindexesondifferentattributescanalsohelpinspeedingupdifferentqueries,withFILTERconditionsondifferentattributes.

Itisoftenbeneficialtocreateanindexonmorethanjustoneattribute.Byaddingmoreattributestoanindex,anindexcanbecomemoreselectiveandthusreducethenumberofdocumentsthatqueriesneedtoprocess.

ArangoDB'sprimaryindexes,edgesindexesandhashindexeswillautomaticallyprovideselectivityestimates.Indexselectivityestimatesareprovidedinthewebinterface,thegetIndexes()returnvalueandintheexplain()outputforagivenquery.

Themoreselectiveanindexis,themoredocumentsitwillfilteronaverage.Theindexselectivityestimatesarethereforeusedbytheoptimizerwhencreatingqueryexecutionplanswhentherearemultipleindexestheoptimizercanchoosefrom.Theoptimizerwillthenselectacombinationofindexeswiththelowestestimatedtotalcost.Ingeneral,theoptimizerwillpicktheindexeswiththehighestestimatedselectivity.

Sparseindexesmayormaynotbepickedbytheoptimizerinaquery.Assparseindexesdonotcontainnullvalues,theywillnotbeusedforqueriesiftheoptimizercannotsafelydeterminewhetheraFILTERconditionincludesnullvaluesfortheindexattributes.Theoptimizerpolicyistoproducecorrectresults,regardlessofwhetherorwhichindexisusedtosatisfyFILTERconditions.Ifitisunsureaboutwhetherusinganindexwillviolatethepolicy,itwillnotmakeuseoftheindex.

Troubleshooting

WhenindoubtaboutwhetherandwhichindexeswillbeusedforexecutingagivenAQLquery,clicktheExplainbuttoninthewebinterfaceintheQueriesvieworusetheexplain()methodforthestatementasfollows(fromtheArangoShell):

varquery="FORdocINcollectionFILTERdoc.value>42RETURNdoc";

varstmt=db._createStatement(query);

stmt.explain();

Theexplain()commandwillreturnadetailedJSONrepresentationofthequery'sexecutionplan.TheJSONexplainoutputisintendedtobeusedbycode.Togetahuman-readableandmuchmorecompactexplanationofthequery,thereisanexplainertool:

varquery="FORdocINcollectionFILTERdoc.value>42RETURNdoc";

require("@arangodb/aql/explainer").explain(query);

Ifanyoftheexplainmethodsshowsthataqueryisnotusingindexes,thefollowingstepsmayhelp:

checkiftheattributenamesinthequeryarecorrectlyspelled.Inaschema-freedatabase,documentsinthesamecollectioncanhavevaryingstructures.Thereisnosuchthingasanon-existingattributeerror.Aquerythatreferstoattributenamesnotpresentinanyofthedocumentswillnotreturnanerror,andobviouslywillnotbenefitfromindexes.

checkthereturnvalueofthegetIndexes()methodforthecollectionsusedinthequeryandvalidatethatindexesareactuallypresentontheattributesusedinthequery'sfilterconditions.

ifindexesarepresentbutnotusedbythequery,theindexesmayhavethewrongtype.Forexample,ahashindexwillonlybeusedforequalitycomparisons(i.e.==)butnotforothercomparisontypessuchas<,<=,>,>=.Additionallyhashindexeswillonlybeusedifalloftheindexattributesareusedinthequery'sFILTERconditions.AskiplistindexwillonlybeusedifatleastitsfirstattributeisusedinaFILTERcondition.Ifadditionallyoftheskiplistindexattributesarespecifiedinthequery(fromleft-to-right),theymayalsobeusedandallowtofiltermoredocuments.

usingindexedattributesasfunctionparametersorinarbitraryexpressionswilllikelyleadtotheindexontheattributenotbeingused.Forexample,thefollowingquerieswillnotuseanindexonvalue:

IndexUtilization

FORdocINcollectionFILTERTO_NUMBER(doc.value)==42RETURNdoc

FORdocINcollectionFILTERdoc.value-1==42RETURNdoc

Inthesecasesthequeriesshouldberewrittensothatonlytheindexattributeispresentononesideoftheoperator,oradditionalfiltersandindexesshouldbeusedtorestricttheamountofdocumentsotherwise.

certainAQLfunctionssuchasWITHIN()orFULLTEXT()doutilizeindexesinternally,buttheiruseisnotmentionedinthequeryexplanationforfunctionsingeneral.Thesefunctionswillraisequeryerrors(atruntime)ifnosuitableindexispresentforthecollectioninquestion.

thequeryoptimizerwillingeneralpickoneindexpercollectioninaquery.ItcanpickmorethanoneindexpercollectioniftheFILTERconditioncontainsmultiplebranchescombinedwithlogicalOR.Forexample,thefollowingqueriescanuseindexes:

FORdocINcollectionFILTERdoc.value1==42||doc.value1==23RETURNdoc

FORdocINcollectionFILTERdoc.value1<42||doc.value2>23RETURNdoc

ThetwoORsinthefirstquerywillbeconvertedtoanINlist,andifthereisasuitableindexonvalue1,itwillbeused.Thesecondqueryrequirestwoseparateindexesonvalue1andvalue2andwillusethemifpresent.Thethirdquerycanusetheindexesonvalue1andvalue2whentheyaresorted.

IndexUtilization

WorkingwithIndexes

IndexIdentifiersandHandles

Anindexhandleuniquelyidentifiesanindexinthedatabase.Itisastringandconsistsofthecollectionnameandanindexidentifierseparatedbya/.Theindexidentifierpartisanumericvaluethatisauto-generatedbyArangoDB.

Aspecificindexofacollectioncanbeaccessedusingitsindexhandleorindexidentifierasfollows:

db.collection.index("<index-handle>");

db.collection.index("<index-identifier>");

db._index("<index-handle>");

Forexample:Assumethattheindexhandle,whichisstoredinthe_idattributeoftheindex,isdemo/362549736andtheindexwascreatedinacollectionnameddemo.Thenthisindexcanbeaccessedas:

db.demo.index("demo/362549736");

Becausetheindexhandleisuniquewithinthedatabase,youcanleaveoutthecollectionandusetheshortcut:

db._index("demo/362549736");

CollectionMethods

Listingallindexesofacollection

returnsinformationabouttheindexesgetIndexes()

Returnsanarrayofallindexesdefinedforthecollection.

Notethat_keyimplicitlyhasanindexassignedtoit.

arangosh>db.test.ensureHashIndex("hashListAttribute",

........>"hashListSecondAttribute.subAttribute");

arangosh>db.test.getIndexes();

Creatinganindex

IndexesshouldbecreatedusingthegeneralmethodensureIndex.Thismethodobsoletesthespecializedindex-specificmethodsensureHashIndex,ensureSkiplist,ensureUniqueConstraintetc.

ensuresthatanindexexistscollection.ensureIndex(index-description)

Ensuresthatanindexaccordingtotheindex-descriptionexists.Anewindexwillbecreatedifnoneexistswiththegivendescription.

Theindex-descriptionmustcontainatleastatypeattribute.Otherattributesmaybenecessary,dependingontheindextype.

typecanbeoneofthefollowingvalues:

hash:hashindexskiplist:skiplistindexfulltext:fulltextindexgeo1:geoindex,withoneattributegeo2:geoindex,withtwoattributes

WorkingwithIndexes

sparsecanbetrueorfalse.

Forhash,andskiplistthesparsitycanbecontrolled,fulltextandgeoaresparsebydefinition.

uniquecanbetrueorfalseandissupportedbyhashorskiplist

Callingthismethodreturnsanindexobject.WhetherornottheindexobjectexistedbeforethecallisindicatedinthereturnattributeisNewlyCreated.

deduplicatecanbetrueorfalseandissupportedbyarrayindexesoftypehashorskiplist.Itcontrolswhetherinsertingduplicateindexvaluesfromthesamedocumentintoauniquearrayindexwillleadtoauniqueconstrainterrorornot.Thedefaultvalueistrue,soonlyasingleinstanceofeachnon-uniqueindexvaluewillbeinsertedintotheindexperdocument.Tryingtoinsertavalueintotheindexthatalreadyexistsintheindexwillalwaysfail,regardlessofthevalueofthisattribute.

Examples

arangosh>db.test.ensureIndex({type:"hash",fields:["a"],sparse:true});

arangosh>db.test.ensureIndex({type:"hash",fields:["a","b"],unique:true});

Droppinganindex

dropsanindexcollection.dropIndex(index)

Dropstheindex.Iftheindexdoesnotexist,thenfalseisreturned.Iftheindexexistedandwasdropped,thentrueisreturned.Notethatyoucannotdropsomespecialindexes(e.g.theprimaryindexofacollectionortheedgeindexofanedgecollection).

collection.dropIndex(index-handle)

Sameasabove.Insteadofanindexanindexhandlecanbegiven.

arangosh>db.example.ensureSkiplist("a","b");

arangosh>varindexInfo=db.example.getIndexes();

arangosh>indexInfo;

arangosh>db.example.dropIndex(indexInfo[0])

arangosh>db.example.dropIndex(indexInfo[1].id)

arangosh>indexInfo=db.example.getIndexes();

LoadIndexesintoMemory

LoadsallindexesofthiscollectionintoMemory.collection.loadIndexesIntoMemory()

Thisfunctiontriestocacheallindexentriesofthiscollectionintothemainmemory.Thereforeititeratesoverallindexesofthecollectionandstorestheindexedvalues,nottheentiredocumentdata,inmemory.Alllookupsthatcouldbefoundinthecachearemuchfasterthanlookupsnotstoredinthecachesoyougetaniceperformanceboost.Itisalsoguaranteedthatthecacheisconsistentwiththestoreddata.

ForthetimebeingthisfunctionisonlyusefulonRocksDBstorageengine,asinMMFilesengineallindexesareinmemoryanyways.

OnRocksDBthisfunctionhonorsallmemorylimits,iftheindexesyouwanttoloadaresmallerthanyourmemorylimitthisfunctionguaranteesthatmostindexvaluesarecached.Iftheindexislargerthanyourmemorylimitthisfunctionwillfillupvaluesuptothislimitandforthetimebeingthereisnowaytocontrolwhichindexesofthecollectionshouldhavepriorityoverothers.

arangosh>db.example.loadIndexesIntoMemory();

"result":true

WorkingwithIndexes

DatabaseMethods

Fetchinganindexbyhandle

findsanindexdb._index(index-handle)

Returnstheindexwithindex-handleornullifnosuchindexexists.

arangosh>db.example.ensureIndex({type:"skiplist",fields:["a","b"]});

arangosh>varindexInfo=db.example.getIndexes().map(function(x){returnx.id;});

arangosh>indexInfo;

arangosh>db._index(indexInfo[0])

arangosh>db._index(indexInfo[1])

Droppinganindex

dropsanindexdb._dropIndex(index)

Dropstheindex.Iftheindexdoesnotexist,thenfalseisreturned.Iftheindexexistedandwasdropped,thentrueisreturned.

db._dropIndex(index-handle)

Dropstheindexwithindex-handle.

arangosh>varindexInfo=db.example.getIndexes();

arangosh>indexInfo;

arangosh>db._dropIndex(indexInfo[0])

arangosh>db._dropIndex(indexInfo[1].id)

arangosh>indexInfo=db.example.getIndexes();

Revalidatingwhetheranindexisused

findsanindex

Soyou'vecreatedanindex,andsinceitsmaintainanceisn'tforfree,youdefinitelywanttoknowwhetheryourquerycanutilizeit.

Youcanuseexplaintoverifywhetherskiplistsorhashindexesareused(ifyouomitcolors:falseyouwillgetnicecolorsinArangoShell):

arangosh>varexplain=require("@arangodb/aql/explainer").explain;

arangosh>explain("FORdocINexampleFILTERdoc.a<23RETURNdoc",{colors:false});

WorkingwithIndexes

HashIndexes

IntroductiontoHashIndexes

Itispossibletodefineahashindexononeormoreattributes(orpaths)ofadocument.ThishashindexisthenusedinqueriestolocatedocumentsinO(1)operations.Ifthehashindexisunique,thennotwodocumentsareallowedtohavethesamesetofattributevalues.

Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheindexisdeclaredsparse,adocumentwillbeexcludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueofnull.

AccessingHashIndexesfromtheShell

UniqueHashIndexes

Ensuresthatauniqueconstraintexists:collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],unique:true})

Createsauniquehashindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.

Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocumentwillwillfailiftheattributeuniquenessisviolated.

Tocreateasparseuniqueindex,setthesparseattributetotrue:

collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],unique:true,sparse:true})

Incasethattheindexwassuccessfullycreated,theindexidentifierisreturned.

Non-existingattributeswilldefaulttonull.Inasparseindexalldocumentswillbeexcludedfromtheindexforwhichallspecifiedindexattributesarenull.Suchdocumentswillnotbetakenintoaccountforuniquenesschecks.

Inanon-sparseindex,alldocumentsregardlessofnull-attributeswillbeindexedandwillbetakenintoaccountforuniquenesschecks.

Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.

arangosh>db.test.ensureIndex({type:"hash",fields:["a","b.c"],unique:true});

arangosh>db.test.save({a:1,b:{c:1}});

arangosh>db.test.save({a:1,b:{c:null}});

arangosh>db.test.save({a:1});

Non-uniqueHashIndexes

Ensuresthatanon-uniquehashindexexists:collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"]})

Createsanon-uniquehashindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.

collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],sparse:true})

arangosh>db.test.ensureIndex({type:"hash",fields:["a"]});

HashIndexes

arangosh>db.test.save({a:null});

HashArrayIndexes

Ensuresthatahasharrayindexexists(non-unique):collection.ensureIndex({type:"hash",fields:["field1[*]",...,"fieldn[*]"]})

Createsanon-uniquehasharrayindexfortheindividualelementsofthearrayattributesfield1[*],...fieldn[*]foundinthedocuments.Atleastoneattributepathhastobegiven.Theindexalwaystreatstheindexedarraysassparse.

Itispossibletocombinearrayindexingwithstandardindexing:collection.ensureIndex({type:"hash",fields:["field1[*]","field2"]})

arangosh>db.test.ensureIndex({type:"hash",fields:["a[*]"]});

arangosh>db.test.save({a:[1,2]});

Ensureuniquenessofrelationsinedgecollections

Itispossibletocreatesecondaryindexesusingtheedgeattributes_fromand_to,startingwithArangoDB3.0.Acombinedindexoverbothfieldstogetherwiththeuniqueoptionenabledcanbeusedtopreventduplicaterelationsfrombeingcreated.

Forexample,adocumentcollectionvertsmightcontainverticeswiththedocumenthandlesverts/A,verts/Bandverts/C.Relationsbetweenthesedocumentscanbestoredinanedgecollectionedgesforinstance.Now,youmaywanttomakesurethatthevertexverts/Aisneverlinkedtoverts/Bbyanedgemorethanonce.Thiscanbeachievedbyaddingaunique,non-sparsehashindexforthefields_fromand_to:

db.edges.ensureIndex({type:"hash",fields:["_from","_to"],unique:true});

Creatinganedge{_from:"verts/A",_to:"verts/B"}inedgeswillbeaccepted,butonlyonce.AnotherattempttostoreanedgewiththerelationA→Bwillberejectedbytheserverwithauniqueconstraintviolatederror.Thisincludesupdatestothe_fromand_tofields.

NotethataddingarelationB→Aisstillpossible,soisA→AandB→B,becausetheyarealldifferentrelationsinadirectedgraph.Eachonecanonlyoccuroncehowever.

HashIndexes

Skiplists

IntroductiontoSkiplistIndexes

ThisisanintroductiontoArangoDB'sskiplists.

Itispossibletodefineaskiplistindexononeormoreattributes(orpaths)ofdocuments.Thisskiplististhenusedinqueriestolocatedocumentswithinagivenrange.Iftheskiplistisdeclaredunique,thennotwodocumentsareallowedtohavethesamesetofattributevalues.

Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheskiplistindexisdeclaredsparse,adocumentwillbeexcludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueofnull.

AccessingSkiplistIndexesfromtheShell

UniqueSkiplistIndex

Ensuresthatauniqueskiplistindexexists:collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],unique:true})

Createsauniqueskiplistindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.

Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocumentwillfailiftheattributeuniquenessisviolated.

collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],unique:true,sparse:true})

Inasparseindexalldocumentswillbeexcludedfromtheindexthatdonotcontainatleastoneofthespecifiedindexattributesorthathaveavalueofnullinanyofthespecifiedindexattributes.Suchdocumentswillnotbeindexed,andnotbetakenintoaccountforuniquenesschecks.

Inanon-sparseindex,thesedocumentswillbeindexed(fornon-presentindexedattributes,avalueofnullwillbeused)andwillbetakenintoaccountforuniquenesschecks.

arangosh>db.ids.ensureIndex({type:"skiplist",fields:["myId"],unique:true});

arangosh>db.ids.save({"myId":123});

arangosh>db.ids.ensureIndex({type:"skiplist",fields:["name.first","name.last"],

unique:true});

arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});

arangosh>db.ids.save({"name":{"first":"jens","last":"jensen"}});

arangosh>db.ids.save({"name":{"first":"hans","last":"jensen"}});

Non-uniqueSkiplistIndex

Skiplists

Ensuresthatanon-uniqueskiplistindexexists:collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"]})

Createsanon-uniqueskiplistindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.

Tocreateasparsenon-uniqueindex,setthesparseattributetotrue.

collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],sparse:true})

arangosh>db.names.ensureIndex({type:"skiplist",fields:["first"]});

arangosh>db.names.save({"first":"Tim"});

arangosh>db.names.save({"first":"Tom"});

arangosh>db.names.save({"first":"John"});

SkiplistArrayIndex

Ensuresthataskiplistarrayindexexists(non-unique):collection.ensureIndex({type:"skiplist",fields:["field1[*]",...,"fieldn[*]"]})

Createsanon-uniqueskiplistarrayindexfortheindividualelementsofthearrayattributesfield1[*],...fieldn[*]foundinthedocuments.Atleastoneattributepathhastobegiven.Theindexalwaystreatstheindexedarraysassparse.

Itispossibletocombinearrayindexingwithstandardindexing:collection.ensureIndex({type:"skiplist",fields:["field1[*]","field2"]})

arangosh>db.test.ensureIndex({type:"skiplist",fields:["a[*]"]});

Querybyexampleusingaskiplistindex

Constructsaquery-by-exampleusingaskiplistindex:collection.byExample(example)

Selectsalldocumentsfromthecollectionthatmatchthespecifiedexampleandreturnsacursor.Askiplistindexwillbeusedifpresent.

{"a":{"c":1}}

{"a":{"c":1},"b":1}

{"a":{"c":1,"b":1}}

Skiplists

willnot.

However,ifyouuse

{"a.c":1},

{"a":{"c":1},"b":1}

{"a":{"c":1,"b":1}}

willmatch.

Skiplists

Persistentindexes

IntroductiontoPersistentIndexes

ThisisanintroductiontoArangoDB'spersistentindexes.

Itispossibletodefineapersistentindexononeormoreattributes(orpaths)ofdocuments.Theindexisthenusedinqueriestolocatedocumentswithinagivenrange.Iftheindexisdeclaredunique,thennotwodocumentsareallowedtohavethesamesetofattributevalues.

Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheindexisdeclaredsparse,adocumentwillbeexcludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueofnull.

AccessingPersistentIndexesfromtheShell

ensuresthatauniquepersistentindexexistscollection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"],unique:true})

Createsauniquepersistentindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.

Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocumentwillwillfailiftheattributeuniquenessisviolated.

collection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"],unique:true,sparse:true})

Inasparseindexalldocumentswillbeexcludedfromtheindexthatdonotcontainatleastoneofthespecifiedindexattributesorthathaveavalueofnullinanyofthespecifiedindexattributes.Suchdocumentswillnotbeindexed,andnotbetakenintoaccountforuniquenesschecks.

Inanon-sparseindex,thesedocumentswillbeindexed(fornon-presentindexedattributes,avalueofnullwillbeused)andwillbetakenintoaccountforuniquenesschecks.

arangosh>db.ids.ensureIndex({type:"persistent",fields:["myId"],unique:true});

arangosh>db.ids.ensureIndex({type:"persistent",fields:["name.first","name.last"],

unique:true});

arangosh>db.ids.save({"name":{"first":"jens","last":"jensen"}});

arangosh>db.ids.save({"name":{"first":"hans","last":"jensen"}});

showexecutionresultsensuresthatanon-uniquepersistentindexexistscollection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"]})

Createsanon-uniquepersistentindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.

Persistent

Tocreateasparseuniqueindex,setthesparseattributetotrue.

arangosh>db.names.ensureIndex({type:"persistent",fields:["first"]});

arangosh>db.names.save({"first":"John"});

Querybyexampleusingapersistentindex

constructsaquery-by-exampleusingapersistentindexcollection.byExample(example)

Selectsalldocumentsfromthecollectionthatmatchthespecifiedexampleandreturnsacursor.Apersistentindexwillbeusedifpresent.

{"a":{"c":1}}

{"a":{"c":1},"b":1}

{"a":{"c":1,"b":1}}

willnot.

However,ifyouuse

{"a.c":1},

{"a":{"c":1},"b":1}

{"a":{"c":1,"b":1}}

willmatch.

PersistentIndexesandServerLanguageTheorderofindexentriesinpersistentindexesadherestotheconfiguredserverlanguage.If,however,theserverisrestartedwithadifferentlanguagesettingaswhenthepersistentindexwascreated,notalldocumentsmaybereturnedanymoreandthesortorderofthosewhicharereturnedcanbewrong(wheneverthepersistentindexisconsulted).

Tofixpersistentindexesafteralanguagechange,deleteandre-createthem.Skiplistindexesarenotaffected,becausetheyarenotpersistedandautomaticallyrebuiltoneveryserverstart.

Persistent

FulltextindexesThisisanintroductiontoArangoDB'sfulltextindexes.

IntroductiontoFulltextIndexes

Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.

Afulltextindexcanbedefinedononeattributeonly,andwillincludeallwordscontainedindocumentsthathaveatextualvalueintheindexattribute.SinceArangoDB2.6theindexwillalsoincludewordsfromtheindexattributeiftheindexattributeisanarrayofstrings,oranobjectwithstringvaluemembers.

Forexample,givenafulltextindexonthetranslationsattributeandthefollowingdocuments,thensearchingforлисаusingthefulltextindexwouldreturnonlythefirstdocument.SearchingfortheindexfortheexactstringFoxwouldreturnthefirsttwodocuments,andsearchingforprefix:Foxwouldreturnallthreedocuments:

{translations:{en:"fox",de:"Fuchs",fr:"renard",ru:"лиса"}}

{translations:"FoxistheEnglishtranslationoftheGermanwordFuchs"}

{translations:["ArangoDB","document","database","Foxx"]}

Notethatdeepernestedobjectsareignored.Forexample,afulltextindexontranslationswouldindexFuchs,butnotfox,giventhefollowingdocumentstructure:

{translations:{en:{US:"fox"},de:"Fuchs"}

Ifyouneedtosearchacrossmultiplefieldsand/ornestedobjects,youmaywriteallthestringsintoaspecialattribute,whichyouthencreatetheindexon(itmightbenecessarytocleanthestringsfirst,e.g.removelinebreaksandstripcertainwords).

Iftheindexattributeisneitherastring,anobjectoranarray,itscontentswillnotbeindexed.Whenindexingthecontentsofanarrayattribute,anarraymemberwillonlybeincludedintheindexifitisastring.Whenindexingthecontentsofanobjectattribute,anobjectmembervaluewillonlybeincludedintheindexifitisastring.Otherdatatypesareignoredandnotindexed.

Currently,fulltextindexesarenotyetsupportedwiththeRocksDBstorageengine.ThusthefunctionFULLTEXT()willbeunavailablewhenusingthisstorageengine.Tousefulltextindexes,pleaseusetheMMFilesstorageengineforthetimebeing.

AccessingFulltextIndexesfromtheShell

Ensuresthatafulltextindexexists:

collection.ensureIndex({type:"fulltext",fields:["field"],minLength:minLength})

Createsafulltextindexonalldocumentsonattributefield.

Fulltextindexesareimplicitlysparse:alldocumentswhichdonothavethespecifiedfieldattributeorthathaveanon-qualifyingvalueintheirfieldattributewillbeignoredforindexing.

Onlyasingleattributecanbeindexed.Specifyingmultipleattributesisunsupported.

TheminimumlengthofwordsthatareindexedcanbespecifiedviatheminLengthparameter.WordsshorterthanminLengthcharacterswillnotbeindexed.minLengthhasadefaultvalueof2,butthisvaluemightbechangedinfutureversionsofArangoDB.Itisthusrecommendedtoexplicitlyspecifythisvalue.

Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetailsisreturned.

arangosh>db.example.ensureIndex({type:"fulltext",fields:["text"],minLength:3});

arangosh>db.example.save({text:"thequickbrown",b:{c:1}});

arangosh>db.example.save({text:"quickbrownfox",b:{c:2}});

arangosh>db.example.save({text:"brownfoxjums",b:{c:3}});

arangosh>db.example.save({text:"foxjumpsover",b:{c:4}});

FulltextIndexes

arangosh>db.example.save({text:"jumpsoverthe",b:{c:5}});

arangosh>db.example.save({text:"overthelazy",b:{c:6}});

arangosh>db.example.save({text:"thelazydog",b:{c:7}});

arangosh>db._query("FORdocumentINFULLTEXT(example,'text','the')RETURNdocument");

showexecutionresultsLooksupafulltextindex:

collection.lookupFulltextIndex(attribute,minLength)

Checkswhetherafulltextindexonthegivenattributeattributeexists.

FulltextAQLFunctions

FulltextAQLfunctionsaredetailedinFulltextfunctions.

FulltextIndexes

GeoIndexes

IntroductiontoGeoIndexes

ThisisanintroductiontoArangoDB'sgeoindexes.

AQL'sgeographicfeaturesaredescribedinGeofunctions.

ArangoDBusesHilbertcurvestoimplementgeo-spatialindexes.Seethisblogfordetails.

Ageo-spatialindexassumesthatthelatitudeisbetween-90and90degreeandthelongitudeisbetween-180and180degree.Ageoindexwillignorealldocumentswhichdonotfulfilltheserequirements.

AccessingGeoIndexesfromtheShell

ensuresthatageoindexexistscollection.ensureIndex({type:"geo",fields:["location"]})

Createsageo-spatialindexonalldocumentsusinglocationaspathtothecoordinates.Thevalueoftheattributehastobeanarraywithatleasttwonumericvalues.Thearraymustcontainthelatitude(firstvalue)andthelongitude(secondvalue).

Alldocuments,whichdonothavetheattributepathorhaveanon-conformingvalueinitareexcludedfromtheindex.

Ageoindexisimplicitlysparse,andthereisnowaytocontrolitssparsity.

Tocreateageoindexonanarrayattributethatcontainslongitudefirst,setthegeoJsonattributetotrue.ThiscorrespondstotheformatdescribedinRFC7946Position

collection.ensureIndex({type:"geo",fields:["location"],geoJson:true})

Tocreateageo-spatialindexonalldocumentsusinglatitudeandlongitudeasseparateattributepaths,twopathsneedtobespecifiedinthefieldsarray:

collection.ensureIndex({type:"geo",fields:["latitude","longitude"]})

Examples

Createageoindexforanarrayattribute:

arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});

........>for(j=-180;j<=180;j+=10){

........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});

........>}

arangosh>db.geo.count();

arangosh>db.geo.near(0,0).limit(3).toArray();

arangosh>db.geo.near(0,0).count();

showexecutionresultsCreateageoindexforahasharrayattribute:

arangosh>db.geo2.ensureIndex({type:"geo",fields:["location.latitude",

"location.longitude"]});

........>for(j=-180;j<=180;j+=10){

........>db.geo2.save({name:"Name/"+i+"/"+j,location:{latitude:i,

longitude:j}});

GeoIndexes

........>}

arangosh>db.geo2.near(0,0).limit(3).toArray();

showexecutionresultsUseGeoIndexwithAQLSORTstatement:

arangosh>db.geoSort.ensureIndex({type:"geo",fields:["latitude","longitude"]});

........>for(j=-180;j<=180;j+=10){

........>db.geoSort.save({name:"Name/"+i+"/"+j,latitude:i,longitude

........>}

arangosh>varquery="FORdocingeoSortSORTDISTANCE(doc.latitude,doc.longitude,0,0)

LIMIT5RETURNdoc"

arangosh>db._explain(query,{},{colors:false});

arangosh>db._query(query);

showexecutionresultsUseGeoIndexwithAQLFILTERstatement:

arangosh>db.geoFilter.ensureIndex({type:"geo",fields:["latitude","longitude"]});

........>for(j=-180;j<=180;j+=10){

........>db.geoFilter.save({name:"Name/"+i+"/"+j,latitude:i,

longitude:j});

........>}

arangosh>varquery="FORdocingeoFilterFILTERDISTANCE(doc.latitude,doc.longitude,

0,0)<2000RETURNdoc"

arangosh>db._explain(query,{},{colors:false});

arangosh>db._query(query);

showexecutionresultsconstructsageoindexselectioncollection.geo(location-attribute)Looksupageoindexdefinedonattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Thisisusefulforcollectionswithmultipledefinedgeoindexes.collection.geo(location_attribute,true)Looksupageoindexonacompoundattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.collection.geo(latitude_attribute,longitude_attribute)Looksupageoindexdefinedonthetwoattributeslatitude_attributeandlongitude-attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thegeosimplequeryhelperfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforrunninggeoqueriesistousetheirAQLequivalents.

Examples

Assumeyouhavealocationstoredaslistintheattributehomeandadestinationstoredintheattributework.Thenyoucanusethegeooperatortoselectwhichgeo-spatialattributes(andthuswhichindex)touseinanearquery.

........>for(j=-180;j<=180;j+=10){

........>db.complex.save({name:"Name/"+i+"/"+j,

........>home:[i,j],

........>work:[-i,-j]});

........>}

GeoIndexes

........>}

........>

arangosh>db.complex.near(0,170).limit(5);

arangosh>db.complex.ensureIndex({type:"geo",fields:["home"]});

arangosh>db.complex.near(0,170).limit(5).toArray();

arangosh>db.complex.geo("work").near(0,170).limit(5);

arangosh>db.complex.ensureIndex({type:"geo",fields:["work"]});

arangosh>db.complex.geo("work").near(0,170).limit(5).toArray();

showexecutionresultsconstructsanearqueryforacollectioncollection.near(latitude,longitude)Thereturnedlistissortedaccordingtothedistance,withthenearestdocumenttothecoordinate(latitude,longitude)comingfirst.Ifthereareneardocumentsofequaldistance,documentsarechosenrandomlyfromthissetuntilthelimitisreached.Itispossibletochangethelimitusingthelimitoperator.Inordertousethenearoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesforthedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.Note:neardoesnotsupportnegativeskips.//However,youcanstilluselimitfollowedtoskip.collection.near(latitude,longitude).limit(limit)Limitstheresulttolimitdocumentsinsteadofthedefault100.Note:Unlikewithmultipleexplicitlimits,limitwillraisetheimplicitdefaultlimitimposedbywithin.collection.near(latitude,longitude).distance()Thiswilladdanattributedistancetoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.collection.near(latitude,longitude).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thenearsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionusingthenearoperatoristousetheAQLNEARfunctioninanAQLqueryasfollows:

FORdocINNEAR(@@collection,@latitude,@longitude,@limit)

RETURNdoc

Examples

Togetthenearesttwolocations:

arangosh>for(vari=-90;i<=90;i+=10){

........>for(varj=-180;j<=180;j+=10){

........>db.geo.save({

........>name:"Name/"+i+"/"+j,

........>loc:[i,j]});

........>}}

showexecutionresultsIfyouneedthedistanceaswell,thenyoucanusethedistanceoperator:

........>for(varj=-180;j<=180;j+=10){

........>name:"Name/"+i+"/"+j,

........>loc:[i,j]});

........>}}

arangosh>db.geo.near(0,0).distance().limit(2).toArray();

showexecutionresultsconstructsawithinqueryforacollectioncollection.within(latitude,longitude,radius)Thiswillfindalldocumentswithinagivenradiusaroundthecoordinate(latitude,longitude).Thereturnedarrayissortedbydistance,beginningwiththenearestdocument.Inordertousethewithinoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesfor

GeoIndexes

thedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.collection.within(latitude,longitude,radius).distance()Thiswilladdanattribute_distancetoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.collection.within(latitude,longitude,radius).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thewithinsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionusingthewithinoperatoristousetheAQLWITHINfunctioninanAQLqueryasfollows:

FORdocINWITHIN(@@collection,@latitude,@longitude,@radius,@distanceAttributeName)

RETURNdoc

Examples

Tofindalldocumentswithinaradiusof2000kmuse:

........>for(varj=-180;j<=180;j+=10){

........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});}}

arangosh>db.geo.within(0,0,2000*1000).distance().toArray();

showexecutionresultsensuresthatageoindexexistscollection.ensureIndex({type:"geo",fields:["location"]})

SinceArangoDB2.5,thismethodisanaliasforensureGeoIndexsincegeoindexesarealwayssparse,meaningthatdocumentsthatdonotcontaintheindexattributesorhavenon-numericvaluesintheindexattributeswillnotbeindexed.ensureGeoConstraintisdeprecatedandensureGeoIndexshouldbeusedinstead.

Theindexdoesnotprovideauniqueoptionbecauseofitslimitedusability.Itwouldpreventidenticalcoordinatesfrombeinginsertedonly,butevenaslightlydifferentlocation(like1inchor1cmoff)wouldbeuniqueagainandnotconsideredaduplicate,althoughitprobablyshould.Thedesiredthresholdfordetectingduplicatesmayvaryforeveryproject(includinghowtocalculatethedistanceeven)andneedstobeimplementedontheapplicationlayerasneeded.YoucanwriteaFoxxserviceforthispurposeandmakeuseoftheAQLgeofunctionstofindnearbycoordinatessupportedbyageoindex.

GeoIndexes

IntroductiontoVertexCentricIndexes

InArangoDBtherearespecialindicesdesignedtospeedupgraphoperations,especiallyifthegraphcontainssupernodes(verticesthathaveanexceptionallyhighamountofconnectededges).Theseindicesarecalledvertexcentricindexesandcanbeusedinadditiontotheexistingedgeindex.

Motivation

Theideaofthisindexistoindexacombinationofavertex,thedirectionandanyarbitrarysetofotherattributesontheedges.Totakeanexample,ifwehaveanattributecalledtypeontheedges,wecanuseanoutboundvertex-centricindexonthisattributetofindalledgesattachedtoavertexwithagiventype.Thefollowingqueryexamplecouldbenefitfromsuchanindex:

FORv,e,pIN3..5OUTBOUND@startGRAPH@graphName

FILTERp.edges[*].typeALL=="friend"

RETURNv

Usingthebuilt-inedge-indexArangoDBcanfindthelistofalledgesattachedtothevertexfast,butstillithastowalkthroughthislistandcheckifallofthemhavetheattributetype=="friend".Usingavertex-centricindexwouldallowArangoDBtofindalledgesforthevertexhavingtheattributetype=="friend"inthesametimeandcansavetheiterationtoverifythecondition.

IndexcreationAvertex-centriccanbeeitherofthefollowingtypes:

HashIndexSkiplistIndexPersistentIndex

Andiscreatedusingtheircreationoperations.Howeverinthelistoffieldsusedtocreatetheindexwehavetoincludeeither_fromor_to.Letusagainexplainthisbyanexample.Assumewewanttocreateanhash-basedoutboundvertex-centricindexontheattributetype.Thiscanbecreatedwiththefollowingway:

arangosh>db.collection.ensureIndex({type:"hash",fields:["_from","type"]})

showexecutionresultsAlloptionsthataresupportedbytherespectiveindexesaresupportedbythevertex-centricindexaswell.

Indexusage

TheAQLoptimizercandecidetouseavertex-centricwheneversuitable,howeveritisnotguaranteedthatthisindexisused,theoptimizermayestimatethatanotherindexisassumedtobebetter.Theoptimizerwillconsiderthistypeofindexesonexplicitfilteringof_fromrespectively_to:

FORedgeINcollection

FILTERedge._from=="vertices/123456"ANDedge.type=="friend"

RETURNedge

andduringpatternmatchingqueries:

FORv,e,pIN3..5OUTBOUND@startGRAPH@graphName

FILTERp.edges[*].typeALL=="friend"

RETURNv

ArangoDBGraphs

FirstStepswithGraphs

AGraphconsistsofverticesandedges.Edgesarestoredasdocumentsinedgecollections.Avertexcanbeadocumentofadocumentcollectionorofanedgecollection(soedgescanbeusedasvertices).Whichcollectionsareusedwithinanamedgraphisdefinedviaedgedefinitions.Anamedgraphcancontainmorethanoneedgedefinition,atleastoneisneeded.Graphsallowyoutostructureyourmodelsinlinewithyourdomainandgroupthemlogicallyincollectionsandgivingyouthepowertoquerytheminthesamegraphqueries.

Newtographs?Takeourfreegraphcourseforfreshersandgetfromzeroknowledgetoadvancedquerytechniques.

Comingfromarelationalbackground-what'sagraph?

InSQLyoucommonlyhavetheconstructofarelationtabletostoren:mrelationsbetweentwodatatables.Anedgecollectionissomewhatsimilartotheserelationtables;vertexcollectionsresemblethedatatableswiththeobjectstoconnect.WhilesimplegraphquerieswithfixednumberofhopsviatherelationtablemaybedoableinSQLwithseveralnestedjoins,graphdatabasescanhandleanarbitrarynumberofthesehopsoveredgecollections-thisiscalledtraversal.Alsoedgesinoneedgecollectionmaypointtoseveralvertexcollections.Itscommontohaveattributesattachedtoedges,i.e.alabelnamingthisinterconnection.Edgeshaveadirection,withtheirrelations_fromand_topointingfromonedocumenttoanotherdocumentstoredinvertexcollections.Inqueriesyoucandefineinwhichdirectionstheedgerelationsmaybefollowed(OUTBOUND:_from→_to,INBOUND:_from←_to,ANY:_from↔_to).

NamedGraphs

NamedgraphsarecompletelymanagedbyArangoDB,andthusalsovisibleinthewebinterface.TheyusethefullspectrumofArangoDB'sgraphfeatures.Youmayaccessthemviaseveralinterfaces.

AQLGraphOperationswithseveralflavors:AQLTraversalsonbothnamedandanonymousgraphsAQLShortestPathonbothnamedandanonymousgraph

JavaScriptGeneralGraphimplementation,asyoumayuseitinFoxxServicesGraphManagement;creating&manipualatinggraphdefinitions;inserting,updatinganddeletingverticesandedgesintographsGraphFunctionsforworkingwithedgesandvertices,toanalyzethemandtheirrelations

JavaScriptSmartGraphimplementation,forscalablegraphsSmartGraphManagement;creating&manipualatingSmartGraphdefinitions;DifferencestoGeneralGraph

RESTfulGeneralGraphinterfaceusedtoimplementgraphmanagementinclientdrivers

Manipulatingcollectionsofnamedgraphswithregulardocumentfunctions

Theunderlyingcollectionsofthenamedgraphsarestillaccessibleusingthestandardmethodsforcollections.Howeverthegraphmoduleaddsanadditionallayerontopofthesecollectionsgivingyouthefollowingguarantees:

AllmodificationsareexecutedtransactionalIfyoudeleteavertexalledgeswillbedeleted,youwillneverhavelooseendsIfyouinsertanedgeitischeckediftheedgematchestheedgedefinitions,youredgecollectionswillonlycontainvalidedges

TheseguaranteesarelostifyouaccessthecollectionsinanyotherwaythanthegraphmoduleorAQL,soifyoudeletedocumentsfromyourvertexcollectionsdirectly,theedgespointingtothemwillberemaininplace.

Anonymousgraphs

Sometimesyoumaynotneedallthepowersofnamedgraphs,butsomeofitsbitsmaybevaluabletoyou.YoumayuseanonymousgraphsinthetraversalsandintheWorkingwithEdgeschapter.Anonymousgraphsdon'thaveedgedefinitionsdescribingwhichvertexcollectionisconnectedbywhichedgecollection.Thegraphmodelhastobemaintainedintheclientsidecode.Thisgivesyoumorefreedomthanthestrictnamedgraphs.

Graphs

AQLGraphOperationsareavailableforboth,namedandanonymousgraphs:AQLTraversalsAQLShortestPath

Whentochooseanonymousornamedgraphs?

Asnotedabove,namedgraphsensuregraphintegrity,bothwheninsertingorremovingedgesorvertices.Soyouwon'tencounterdanglingedges,evenifyouusethesamevertexcollectioninseveralnamedgraphs.Thisinvolvesmoreoperationsinsidethedatabasewhichcomeatacost.Thereforeanonymousgraphsmaybefasterinmanyoperations.Sothisquestionmaybenarroweddownto:'CanIaffordtheadditionaleffortordoIneedthewarrantyforintegrity?'.

Multipleedgecollectionsvs.FILTERsonedgedocumentattributes

Ifyouwanttoonlytraverseedgesofaspecifictype,therearetwowaystoachievethis.Thefirstwouldbeanattributeintheedgedocument-i.e.type,whereyouspecifyadifferentiatorfortheedge-i.e."friends","family","married"or"workmates",soyoucanlaterFILTERe.type="friends"ifyouonlywanttofollowthefriendedges.

Anotherway,whichmaybemoreefficientinsomecases,istousedifferentedgecollectionsfordifferenttypesofedges,soyouhavefriend_edges,family_edges,married_edgesandworkmate_edgesascollectionnames.Youcanthenconfigureseveralnamedgraphsincludingasubsetoftheavailableedgeandvertexcollections-oryouuseanonymousgraphqueries,whereyouspecifyalistofedgecollectionstotakeintoaccountinthatquery.Toonlyfollowfriendedges,youwouldspecifyfriend_edgesassoleedgecollection.

Bothapproacheshaveadvantagesanddisadvantages.FILTERoperationsonedeattributeswilldocomparisonsoneachtraversededge,whichmaybecomeCPU-intense.Whennotfindingtheedgesinthefirstplacebecauseofthecollectioncontainingthemisnottraversedatall,therewillneverbeareasontoactualycheckfortheirtypeattributewithFILTER.

Themultipleedgecollectionsapproachislimitedbythenumberofcollectionsthatcanbeusedsimultaneouslyinonequery.EverycollectionusedinaqueryrequiressomeresourcesinsideofArangoDBandthenumberisthereforelimitedtocaptheresourcerequirements.Youmayalsohaveconstraintsonotheredgeattributes,suchasahashindexwithauniqueconstraint,whichrequiresthedocumentstobeinasinglecollectionfortheuniquenessguarantee,anditmaythusnotbepossibletostorethedifferenttypesofedgesinmultipleedegcollections.

So,ifyouredgeshaveaboutadozendifferenttypes,it'sokaytochoosethecollectionapproach,otherwisetheFILTERapproachispreferred.YoucanstilluseFILTERoperationsonedgesofcourse.YoucangetridofaFILTERonthetypewiththeformerapproach,everythingelsecanstaythesame.

WhichpartofmydataisanEdgeandwhichaVertex?

Themainobjectsinyourdatamodel,suchasusers,groupsorarticles,areusuallyconsideredtobevertices.Foreachtypeofobject,adocumentcollection(alsocalledvertexcollection)shouldstoretheindividualentities.Entitiescanbeconnectedbyedgestoexpressandclassifyrelationsbetweenvertices.Itoftenmakessensetohaveanedgecollectionperrelationtype.

ArangoDBdoesnotrequireyoutostoreyourdataingraphstructureswithedgesandvertices,youcanalsodecidetoembedattributessuchaswhichgroupsauserispartof,or_idsofdocumentsinanotherdocumentinsteadofconnectingthedocumentswithedges.Itcanbeameaningfulperformanceoptimizationfor1:nrelationships,ifyourdataisnotfocusedonrelationsandyoudon'tneedgraphtraversalwithvaryingdepth.Itusuallymeanstointroducesomeredundancyandpossiblyinconsistenciesifyouembeddata,butitcanbeanacceptabletradeoff.

Vertices

Let'ssaywehavetwovertexcollections,UsersandGroups.DocumentsintheGroupscollectioncontaintheattributesoftheGroup,i.e.whenitwasfounded,itssubject,aniconURLandsoon.Usersdocumentscontainthedataspecifictoauser-likeallnames,birthdays,AvatarURLs,hobbies...

Wecanuseanedgecollectiontostorerelationsbetweenusersandgroups.Sincemultipleusersmaybeinanarbitrarynumberofgroups,thisisanm:nrelation.TheedgecollectioncanbecalledUsersInGroupswithi.e.oneedgewith_frompointingtoUsers/Johnand_topointingtoGroups/BowlingGroupHappyPin.ThismakestheuserJohnamemberofthegroupBowlingGroupHappyPin.

Graphs

Attributesofthisrelationmaycontainqualifierstothisrelation,likethepermissionsofJohninthisgroup,thedatewhenhejoinedthegroupetc.

Soroughlyput,ifyouusedocumentsandtheirattributesinasentence,nounswouldtypicallybevertices,verbsbecometheedges.Youcanseethisintheknowsgraphbelow:

AliceknowsBob,whointermknowsCharlie.

Advantagesofthisapproach

Graphsgiveyoutheadvantageofnotjustbeingabletohaveafixednumberofm:nrelationsinarow,butanarbitrarynumber.Edgescanbetraversedinbothdirections,soit'seasytodetermineallgroupsauserisin,butalsotofindoutwhichmembersacertaingrouphas.Userscouldalsobeinterconnectedtocreateasocialnetwork.

Usingthegraphdatamodel,dealingwithdatathathaslotsofrelationsstaysmanageableandcanbequeriedinveryflexibleways,whereasitwouldcauseheadachetohandleitinarelationaldatabasesystem.

Backupandrestore

Forsureyouwanttohavebackupsofyourgraphdata,youcanuseArangodumptocreatethebackup,andArangorestoretorestoreabackupintoanewArangoDB.Youshouldhowevernotethat:

youneedthesystemcollection_graphsifyoubackupnamedgraphs.youneedtobackupthecompletesetofalledgeandvertexcollectionsyourgraphconsistsof.Partialdump/restoremaynotwork.

Managinggraphs

Bydefaultyoushouldusetheinterfaceyourdriverprovidestomanagegraphs.

Thisisi.e.documentedinGraphs-SectionoftheArangoDBJavadriver.

ExampleGraphs

ArangoDBcomeswithasetofeasilygraspablegraphsthatareusedtodemonstratetheAPIs.Youcanusetheaddsamplestabinthecreategraphwindowinthewebinterface,orloadthemodule@arangodb/graph-examples/example-graphinarangoshanduseittocreateinstancesofthesegraphsinyourArangoDB.Onceyou'vecreatedthem,youcaninspecttheminthewebinterface-whichwasusedtocreatethepicturesbelow.

Youcaneasilylookintotheinnardsofthisscriptforreferenceabouthowtomanagegraphsprogramatically.

TheKnows_Graph

Graphs

Asetofpersonsknowingeachother:

Theknowsgraphconsistsofonevertexcollectionpersonsconnectedviaoneedgecollectionknows.ItwillcontainfivepersonsAlice,Bob,Charlie,DaveandEve.Wewillhavethefollowingdirectedrelations:

AliceknowsBobBobknowsCharlieBobknowsDaveEveknowsAliceEveknowsBob

Thisishowwecreateit,inspectitsverticesandedges,anddropitagain:

arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");

arangosh>varg=examples.loadGraph("knows_graph");

arangosh>db.persons.toArray()

arangosh>db.knows.toArray();

arangosh>examples.dropGraph("knows_graph");

TheSocialGraph

Asetofpersonsandtheirrelations:

Graphs

Thisexamplehasfemaleandmalepersonsasverticesintwovertexcollections-femaleandmale.Theedgesaretheirconnectionsintherelationedgecollection.Thisishowwecreateit,inspectitsverticesandedges,anddropitagain:

arangosh>vargraph=examples.loadGraph("social");

arangosh>db.female.toArray()

arangosh>db.male.toArray()

arangosh>db.relation.toArray()

arangosh>examples.dropGraph("social");

TheCityGraph

Asetofeuropeancities,andtheirfictionaltravelingdistancesasconnections:

Graphs

Theexamplehasthecitiesasverticesinseveralvertexcollections-germanCityandfrenchCity.Theedgesaretheirinterconnectionsinseveraledgecollectionsfrench/german/internationalHighway.Thisishowwecreateit,inspectitsedgesandvertices,anddropitagain:

arangosh>varg=examples.loadGraph("routeplanner");

arangosh>db.frenchCity.toArray();

arangosh>db.germanCity.toArray();

arangosh>db.germanHighway.toArray();

arangosh>db.frenchHighway.toArray();

arangosh>db.internationalHighway.toArray();

arangosh>examples.dropGraph("routeplanner");

TheTraversalGraph

Thisgraphwasdesignedtodemonstratefiltersintraversals.Ithassomelabelstofilteronit.

Graphs

Theexamplehasallitsverticesinthecirclescollection,andanedgesedgecollectiontoconnectthem.Circleshaveuniquenumericlabels.Edgeshavetwobooleanattributes(theFalsealwaysbeingfalse,theTruthalwaysbeingtrue)andalabelsortingB-Dtotheleftside,G-Ktotherightside.LeftandrightsidesplitintoPaths-atBandGwhichareeachdirectneighboursoftheroot-nodeA.StartingfromAthegraphhasadepthof3onallitspaths.

arangosh>varg=examples.loadGraph("traversalGraph");

arangosh>db.circles.toArray();

arangosh>db.edges.toArray();

arangosh>examples.dropGraph("traversalGraph");

TheWorldGraph

Theworldcountrygraphstructuresitsnodeslikethat:world→continent→country→capital.Insomecasesedgedirectionsaren'tforward(thereforeitwillbedisplayeddisjunctinthegraphviewer).Ithastwowaysofcreatingit.Oneusingthenamedgraphutilities(worldCountry),onewithout(worldCountryUnManaged).Itisusedtodemonstraterawtraversaloperations.

arangosh>varg=examples.loadGraph("worldCountry");

arangosh>db.worldVertices.toArray();

arangosh>db.worldEdges.toArray();

arangosh>examples.dropGraph("worldCountry");

arangosh>varg=examples.loadGraph("worldCountryUnManaged");

arangosh>examples.dropGraph("worldCountryUnManaged");

Graphs

Cookbookexamples

TheabovereferencedchaptersdescribethevariousAPIsofArangoDBsgraphenginewithsmallexamples.Ourcookbookhassomemorereallifeexamples:

TraversingagraphinfulldepthUsinganexamplevertexwiththejavadriverRetrievingdocumentsfromArangoDBwithoutknowingthestructureUsingacustomvisitorfromnode.jsAQLExampleQueriesonanActorsandMoviesDatabase

Highervolumegraphexamples

Alloftheaboveexamplesarerathersmallsotheyareeasiertocomprehendandcandemonstratethewaythefunctionalityworks.Therearehoweverseveraldatasetsfreelyavailableonthewebthatarealotbigger.Wecollectedsomeofthemwithimportscriptssoyoumayplayaroundwiththem.AnotherhugegraphisthePokecsocialnetworkfromSlovakiathatweusedforperformancetestingonseveraldatabases;Youwillfindimportingscriptsetc.inthisblogpost.

Graphs

GraphsThischapterdescribesthegeneral-graphmodule.Itallowsyoutodefineagraphthatisspreadacrossseveraledgeanddocumentcollections.Thisallowsyoutostructureyourmodelsinlinewithyourdomainandgroupthemlogicallyincollectionsgivingyouthepowertoquerytheminthesamegraphqueries.Thereisnoneedtoincludethereferencedcollectionswithinthequery,thismodulewillhandleitforyou.

ThreeStepstocreateagraph

Createagraph

arangosh>vargraph_module=require("@arangodb/general-graph");

arangosh>vargraph=graph_module._create("myGraph");

arangosh>graph;

{[Graph]

Addsomevertexcollections

arangosh>graph._addVertexCollection("shop");

arangosh>graph._addVertexCollection("customer");

arangosh>graph._addVertexCollection("pet");

arangosh>graph;

showexecutionresultsDefinerelationsontheGraph

arangosh>varrel=graph_module._relation("isCustomer",["shop"],["customer"]);

arangosh>graph._extendEdgeDefinitions(rel);

arangosh>graph;

GeneralGraphs

GraphManagementThischapterdescribesthejavascriptinterfaceforcreatingandmodifyingnamedgraphs.Inordertocreateanonemptygraphthefunctionalitytocreateedgedefinitionshastobeintroducedfirst:

EdgeDefinitions

Anedgedefinitionisalwaysadirectedrelationofagraph.Eachgraphcanhavearbitrarymanyrelationsdefinedwithintheedgedefinitionsarray.

Initializethelist

Createalistofedgedefinitionstoconstructagraph.

graph_module._edgeDefinitions(relation1,relation2,...,relationN)

Thelistofedgedefinitionsofagraphcanbemanagedbythegraphmoduleitself.Thisfunctionistheentrypointforthemanagementandwillreturnthecorrectlist.

Parameters

relationX(optional)Anobjectrepresentingadefinitionofonerelationinthegraph

Examples

arangosh>directed_relation=graph_module._relation("lives_in","user","city");

arangosh>undirected_relation=graph_module._relation("knows","user","user");

arangosh>edgedefinitions=graph_module._edgeDefinitions(directed_relation,

undirected_relation);

Extendthelist

Extendthelistofedgedefinitionstoconstructagraph.

graph_module._extendEdgeDefinitions(edgeDefinitions,relation1,relation2,...,relationN)

Inordertoaddmoreedgedefinitionstothegraphbeforecreatingthisfunctioncanbeusedtoaddmoredefinitionstotheinitiallist.

Parameters

edgeDefinitions(required)Alistofrelationdefinitionobjects.relationX(required)Anobjectrepresentingadefinitionofonerelationinthegraph

Examples

arangosh>directed_relation=graph_module._relation("lives_in","user","city");

arangosh>undirected_relation=graph_module._relation("knows","user","user");

arangosh>edgedefinitions=graph_module._edgeDefinitions(directed_relation);

arangosh>edgedefinitions=graph_module._extendEdgeDefinitions(undirected_relation);

Relation

Defineadirectedrelation.

GraphManagement

graph_module._relation(relationName,fromVertexCollections,toVertexCollections)

TherelationNamedefinesthenameofthisrelationandreferencestotheunderlyingedgecollection.ThefromVertexCollectionsisanArrayofdocumentcollectionsholdingthestartvertices.ThetoVertexCollectionsisanArrayofdocumentcollectionsholdingthetargetvertices.RelationsareonlyallowedinthedirectionfromanycollectioninfromVertexCollectionstoanycollectionintoVertexCollections.

Parameters

relationName(required)Thenameoftheedgecollectionwheretheedgesshouldbestored.Willbecreatedifitdoesnotyetexist.fromVertexCollections(required)Oneoralistofcollectionnames.Sourceverticesfortheedgeshavetobestoredinthesecollections.Collectionswillbecreatediftheydonotexist.toVertexCollections(required)Oneoralistofcollectionnames.Targetverticesfortheedgeshavetobestoredinthesecollections.Collectionswillbecreatediftheydonotexist.

Examples

arangosh>graph_module._relation("has_bought",["Customer","Company"],["Groceries",

"Electronics"]);

arangosh>graph_module._relation("has_bought","Customer","Product");

CreateagraphAfterhavingintroducededgedefinitionsagraphcanbecreated.

Createagraph

graph_module._create(graphName,edgeDefinitions,orphanCollections)

Thecreationofagraphrequiresthenameofthegraphandadefinitionofitsedges.

Foreverytypeofedgedefinitionaconveniencemethodexiststhatcanbeusedtocreateagraph.Optionallyalistofvertexcollectionscanbeadded,whicharenotusedinanyedgedefinition.Thesecollectionsarereferredtoasorphancollectionswithinthischapter.Allcollectionsusedwithinthecreationprocessarecreatediftheydonotexist.

Parameters

graphName(required)UniqueidentifierofthegraphedgeDefinitions(optional)ListofrelationdefinitionobjectsorphanCollections(optional)Listofadditionalvertexcollectionnames

Examples

Createanemptygraph,edgedefinitionscanbeaddedatruntime:

arangosh>graph=graph_module._create("myGraph");

{[Graph]

Createagraphusinganedgecollectionedgesandasinglevertexcollectionvertices

arangosh>varedgeDefinitions=[{collection:"edges","from":["vertices"],"to":[

"vertices"]}];

GraphManagement

arangosh>graph=graph_module._create("myGraph",edgeDefinitions);

{[Graph]

"edges":[ArangoCollection18560,"edges"(typeedge,statusloaded)],

"vertices":[ArangoCollection18555,"vertices"(typedocument,statusloaded)]

Createagraphwithedgedefinitionsandorphancollections:

arangosh>graph=graph_module._create("myGraph",

........>[graph_module._relation("myRelation",["male","female"],["male","female"])],

["sessions"]);

CompleteExampletocreateagraph

ExampleCall:

arangosh>varedgeDefinitions=graph_module._edgeDefinitions();

arangosh>graph_module._extendEdgeDefinitions(edgeDefinitions,

graph_module._relation("friend_of","Customer","Customer"));

arangosh>graph_module._extendEdgeDefinitions(

........>edgeDefinitions,graph_module._relation(

........>"has_bought",["Customer","Company"],["Groceries","Electronics"]));

arangosh>graph_module._create("myStore",edgeDefinitions);

showexecutionresultsalternativecall:

arangosh>varedgeDefinitions=graph_module._edgeDefinitions(

........>graph_module._relation("friend_of",["Customer"],["Customer"]),

graph_module._relation(

........>"has_bought",["Customer","Company"],["Groceries","Electronics"]));

arangosh>graph_module._create("myStore",edgeDefinitions);

Listavailablegraphs

Listallgraphs.

graph_module._list()

Listsallgraphnamesstoredinthisdatabase.

Examples

arangosh>graph_module._list();

Loadagraph

GraphManagement

Getagraph

graph_module._graph(graphName)

Agraphcanberetrievedbyitsname.

Parameters

graphName(required)Uniqueidentifierofthegraph

Examples

Getagraph:

arangosh>graph=graph_module._graph("social");

Removeagraph

graph_module._drop(graphName,dropCollections)

Agraphcanbedroppedbyitsname.Thiscandropallcollectionscontainedinthegraphaslongastheyarenotusedwithinothergraphs.Todropthecollectionsonlybelongingtothisgraph,theoptionalparameterdrop-collectionshastobesettotrue.

Parameters

graphName(required)UniqueidentifierofthegraphdropCollections(optional)Defineifcollectionsshouldbedropped(default:false)

Examples

Dropagraphandkeepcollections:

arangosh>graph_module._drop("social");

arangosh>db._collection("female");

[ArangoCollection18682,"female"(typedocument,statusloaded)]

arangosh>db._collection("male");

[ArangoCollection18685,"male"(typedocument,statusloaded)]

arangosh>db._collection("relation");

[ArangoCollection18688,"relation"(typeedge,statusloaded)]

arangosh>graph_module._drop("social",true);

arangosh>db._collection("female");

arangosh>db._collection("male");

arangosh>db._collection("relation");

ModifyagraphdefinitionduringruntimeAfteryouhavecreatedangraphitsdefinitionisnotimmutable.Youcanstilladd,deleteormodifyedgedefinitionsandvertexcollections.

GraphManagement

Extendtheedgedefinitions

Addanotheredgedefinitiontothegraph

graph._extendEdgeDefinitions(edgeDefinition)

Extendstheedgedefinitionsofagraph.Ifanorphancollectionisusedinthisedgedefinition,itwillberemovedfromtheorphanage.Iftheedgecollectionoftheedgedefinitiontoaddisalreadyusedinthegraphorusedinadifferentgraphwithdifferentfromand/ortocollectionsanerroristhrown.

Parameters

edgeDefinition(required)Therelationdefinitiontoextendthegraph

Examples

arangosh>vargraph_module=require("@arangodb/general-graph")

arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);

arangosh>vargraph=graph_module._create("myGraph",[ed1]);

arangosh>graph._extendEdgeDefinitions(ed2);

Modifyanedgedefinition

Modifyanrelationdefinition

graph_module._editEdgeDefinition(edgeDefinition)

Editsonerelationdefinitionofagraph.Theedgedefinitionusedasargumentwillreplacetheexistingedgedefinitionofthegraphwhichhasthesamecollection.VertexCollectionsofthereplacededgedefinitionthatarenotusedinthenewdefinitionwilltransformtoanorphan.Orphansthatareusedinthisnewedgedefinitionwillbedeletedfromthelistoforphans.Othergraphswiththesameedgedefinitionwillbemodified,too.

Parameters

edgeDefinition(required)Theedgedefinitiontoreplacetheexistingedgedefinitionwiththesameattributecollection.

Examples

arangosh>varoriginal=graph_module._relation("myEC1",["myVC1"],["myVC2"]);

arangosh>varmodified=graph_module._relation("myEC1",["myVC2"],["myVC3"]);

arangosh>vargraph=graph_module._create("myGraph",[original]);

arangosh>graph._editEdgeDefinitions(modified);

Deleteanedgedefinition

Deleteonerelationdefinition

graph_module._deleteEdgeDefinition(edgeCollectionName,dropCollection)

Deletesarelationdefinitiondefinedbytheedgecollectionofagraph.Ifthecollectionsdefinedintheedgedefinition(collection,from,to)arenotusedinanotheredgedefinitionofthegraph,theywillbemovedtotheorphanage.

Parameters

edgeCollectionName(required)Nameofedgecollectionintherelationdefinition.dropCollection(optional)Defineiftheedgecollectionshouldbedropped.Defaultfalse.

Examples

Removeanedgedefinitionbutkeeptheedgecollection:

GraphManagement

arangosh>vargraph=graph_module._create("myGraph",[ed1,ed2]);

arangosh>graph._deleteEdgeDefinition("myEC1");

arangosh>db._collection("myEC1");

[ArangoCollection25989,"myEC1"(typeedge,statusloaded)]

Removeanedgedefinitionanddroptheedgecollection:

arangosh>vargraph=graph_module._create("myGraph",[ed1,ed2]);

arangosh>graph._deleteEdgeDefinition("myEC1",true);

arangosh>db._collection("myEC1");

ExtendvertexCollections

Eachgraphcanhaveanarbitraryamountofvertexcollections,whicharenotpartofanyedgedefinitionofthegraph.Thesecollectionsarecalledorphancollections.Ifthegraphisextendedwithanedgedefinitionusingoneoftheorphans,itwillberemovedfromthesetoforphancollectionautomatically.

Addavertexcollection

Addavertexcollectiontothegraph

graph._addVertexCollection(vertexCollectionName,createCollection)

Addsavertexcollectiontothesetoforphancollectionsofthegraph.Ifthecollectiondoesnotexist,itwillbecreated.Ifitisalreadyusedbyanyedgedefinitionofthegraph,anerrorwillbethrown.

Parameters

vertexCollectionName(required)Nameofvertexcollection.createCollection(optional)Iftruethecollectionwillbecreatedifitdoesnotexist.Default:true.

Examples

arangosh>graph._addVertexCollection("myVC3",true);

Gettheorphanedcollections

Getallorphancollections

graph._orphanCollections()

Returnsallvertexcollectionsofthegraphthatarenotusedinanyedgedefinition.

Examples

arangosh>graph._orphanCollections();

GraphManagement

"myVC3"

Removeavertexcollection

Removeavertexcollectionfromthegraph

graph._removeVertexCollection(vertexCollectionName,dropCollection)

Removesavertexcollectionfromthegraph.Onlycollectionsnotusedinanyrelationdefinitioncanberemoved.Optionallythecollectioncanbedeleted,ifitisnotusedinanyothergraph.

Parameters

vertexCollectionName(required)Nameofvertexcollection.dropCollection(optional)Iftruethecollectionwillbedroppedifitisnotusedinanyothergraph.Default:false.

Examples

arangosh>graph._removeVertexCollection("myVC3");

ManiuplatingVertices

Saveavertex

CreateanewvertexinvertexCollectionName

graph.vertexCollectionName.save(data)

Parameters

data(required)JSONdataofvertex.

Examples

arangosh>graph.male.save({name:"Floyd",_key:"floyd"});

Replaceavertex

ReplacesthedataofavertexincollectionvertexCollectionName

graph.vertexCollectionName.replace(vertexId,data,options)

Parameters

vertexId(required)_idattributeofthevertexdata(required)JSONdataofvertex.

GraphManagement

options(optional)Seecollectiondocumentation

Examples

arangosh>graph.male.save({neym:"Jon",_key:"john"});

arangosh>graph.male.replace("male/john",{name:"John"});

Updateavertex

UpdatesthedataofavertexincollectionvertexCollectionName

graph.vertexCollectionName.update(vertexId,data,options)

Parameters

vertexId(required)_idattributeofthevertexdata(required)JSONdataofvertex.options(optional)Seecollectiondocumentation

Examples

arangosh>graph.female.save({name:"Lynda",_key:"linda"});

arangosh>graph.female.update("female/linda",{name:"Linda",_key:"linda"});

Removeavertex

RemovesavertexincollectionvertexCollectionName

graph.vertexCollectionName.remove(vertexId,options)

Additionallyremovesallingoingandoutgoingedgesofthevertexrecursively(seeedgeremove).

Parameters

vertexId(required)_idattributeofthevertexoptions(optional)Seecollectiondocumentation

Examples

arangosh>graph.male.save({name:"Kermit",_key:"kermit"});

arangosh>db._exists("male/kermit")

arangosh>graph.male.remove("male/kermit")

arangosh>db._exists("male/kermit")

ManipulatingEdges

Saveanewedge

GraphManagement

CreatesanedgefromvertexfromtovertextoincollectionedgeCollectionName

graph.edgeCollectionName.save(from,to,data,options)

Parameters

from(required)_idattributeofthesourcevertexto(required)_idattributeofthetargetvertexdata(required)JSONdataoftheedgeoptions(optional)Seecollectiondocumentation

Examples

arangosh>graph.relation.save("male/bob","female/alice",{type:"married",_key:

"bobAndAlice"});

showexecutionresultsIfthecollectionsoffromandtoarenotdefinedinanedgedefinitionofthegraph,theedgewillnotbestored.

arangosh>graph.relation.save(

........>"relation/aliceAndBob",

........>"female/alice",

........>{type:"married",_key:"bobAndAlice"});

[ArangoError1906:invalidedgebetweenrelation/aliceAndBobandfemale/alice.Doesn't

conformtoanyedgedefinition]

Replaceanedge

ReplacesthedataofanedgeincollectionedgeCollectionName.Notethat_fromand_toaremandatory.

graph.edgeCollectionName.replace(edgeId,data,options)

Parameters

edgeId(required)_idattributeoftheedgedata(required)JSONdataoftheedgeoptions(optional)Seecollectiondocumentation

Examples

arangosh>graph.relation.save("female/alice","female/diana",{typo:"nose",_key:

"aliceAndDiana"});

arangosh>graph.relation.replace("relation/aliceAndDiana",{type:"knows",_from:

"female/alice",_to:"female/diana"});

Updateanedge

UpdatesthedataofanedgeincollectionedgeCollectionName

graph.edgeCollectionName.update(edgeId,data,options)

Parameters

GraphManagement

edgeId(required)_idattributeoftheedgedata(required)JSONdataoftheedgeoptions(optional)Seecollectiondocumentation

Examples

arangosh>graph.relation.save("female/alice","female/diana",{type:"knows",_key:

"aliceAndDiana"});

arangosh>graph.relation.update("relation/aliceAndDiana",{type:"quarreled",_key:

"aliceAndDiana"});

Removeanedge

RemovesanedgeincollectionedgeCollectionName

graph.edgeCollectionName.remove(edgeId,options)

Ifthisedgeisusedasavertexbyanotheredge,theotheredgewillberemoved(recursively).

Parameters

edgeId(required)_idattributeoftheedgeoptions(optional)Seecollectiondocumentation

Examples

arangosh>graph.relation.save("female/alice","female/diana",{_key:"aliceAndDiana"});

arangosh>db._exists("relation/aliceAndDiana")

arangosh>graph.relation.remove("relation/aliceAndDiana")

arangosh>db._exists("relation/aliceAndDiana")

Connectedges

Getallconnectingedgesbetween2groupsofverticesdefinedbytheexamples

graph._getConnectingEdges(vertexExample,vertexExample2,options)

Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.

Parameters

vertexExample1(optional)SeeDefinitionofexamplesvertexExample2(optional)SeeDefinitionofexamplesoptions(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:

edgeExamples:Filtertheedges,seeDefinitionofexamplesedgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.vertex1CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertexsteps.vertex2CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertexsteps.

Examples

Arouteplannerexample,allconnectingedgesbetweencapitals.

GraphManagement

arangosh>vargraph=examples.loadGraph("routeplanner");

arangosh>graph._getConnectingEdges({isCapital:true},{isCapital:true});

GraphManagement

GraphFunctionsThischapterdescribesvariousfunctionsonagraph.Alotoftheseacceptavertex(oredge)exampleasparameterasdefinedinthenextsection.

ExampleswillexplaintheAPIonthethecitygraph:

DefinitionofexamplesFormanyofthefollowingfunctionsexamplescanbepassedinasaparameter.Examplesareusedtofiltertheresultsetforobjectsthatmatchtheconditions.Theseexamplescanhavethefollowingvalues:

null,thereisnomatchingexecutedallfoundresultsarevalid.Astring,onlyresultsarereturned,which_idequalthevalueofthestringAnexampleobject,definingasetofattributes.Onlyresultshavingtheseattributesarematched.Alistcontainingexampleobjectsand/orstrings.Allresultsmatchingatleastoneoftheelementsinthelistarereturned.Getverticesfromedges.

Getvertexfromofanedge

Getthesourcevertexofanedge

graph._fromVertex(edgeId)

Returnsthevertexdefinedwiththeattribute_fromoftheedgewithedgeIdasits_id.

Parameters

edgeId(required)_idattributeoftheedge

Examples

GraphFunctions

arangosh>varany=require("@arangodb").db.relation.any();

arangosh>graph._fromVertex("relation/"+any._key);

Getvertextoofanedge

Getthetargetvertexofanedge

graph._toVertex(edgeId)

Returnsthevertexdefinedwiththeattribute_tooftheedgewithedgeIdasits_id.

Parameters

edgeId(required)_idattributeoftheedge

Examples

arangosh>varany=require("@arangodb").db.relation.any();

arangosh>graph._toVertex("relation/"+any._key);

_neighbors

Getallneighborsoftheverticesdefinedbytheexample

graph._neighbors(vertexExample,options)

Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.ThecomplexityofthismethodisO(n*m^x)withnbeingtheverticesdefinedbytheparametervertexExamplex,mtheaverageamountofneighborsandxthemaximaldepths.HencethedefaultcallwouldhaveacomplexityofO(n*m);

Parameters

vertexExample(optional)SeeDefinitionofexamplesoptions(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:

direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).edgeExamples:Filtertheedges,seeDefinitionofexamplesneighborExamples:Filtertheneighborvertices,seeDefinitionofexamplesedgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.vertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertexsteps.minDepth:Definestheminimalnumberofintermediatestepstoneighbors(defaultis1).maxDepth:Definesthemaximalnumberofintermediatestepstoneighbors(defaultis1).

Examples

Arouteplannerexample,allneighborsofcapitals.

arangosh>graph._neighbors({isCapital:true});

showexecutionresultsArouteplannerexample,alloutboundneighborsofHamburg.

GraphFunctions

arangosh>graph._neighbors('germanCity/Hamburg',{direction:'outbound',maxDepth:2});

_commonNeighbors

Getallcommonneighborsoftheverticesdefinedbytheexamples.

graph._commonNeighbors(vertex1Example,vertex2Examples,optionsVertex1,optionsVertex2)

Thisfunctionreturnstheintersectionofgraph_module._neighbors(vertex1Example,optionsVertex1)andgraph_module._neighbors(vertex2Example,optionsVertex2).Forparameterdocumentationsee_neighbors.

ThecomplexityofthismethodisO(n*m^x)withnbeingthemaximalamountofverticesdefinedbytheparametersvertexExamples,mtheaverageamountofneighborsandxthemaximaldepths.HencethedefaultcallwouldhaveacomplexityofO(n*m);

Examples

Arouteplannerexample,allcommonneighborsofcapitals.

arangosh>graph._commonNeighbors({isCapital:true},{isCapital:true});

showexecutionresultsArouteplannerexample,allcommonoutboundneighborsofHamburgwithanyotherlocationwhichhaveamaximaldepthof2:

arangosh>graph._commonNeighbors(

........>'germanCity/Hamburg',

........>{},

........>{direction:'outbound',maxDepth:2},

........>{direction:'outbound',maxDepth:2});

_countCommonNeighborsGettheamountofcommonneighborsoftheverticesdefinedbytheexamples.

graph._countCommonNeighbors(vertex1Example,vertex2Examples,optionsVertex1,optionsVertex2)

Similarto_commonNeighborsbutreturnscountinsteadoftheelements.

Examples

Arouteplannerexample,allcommonneighborsofcapitals.

arangosh>varexample={isCapital:true};

arangosh>varoptions={includeData:true};

arangosh>graph._countCommonNeighbors(example,example,options,options);

GraphFunctions

Arouteplannerexample,allcommonoutboundneighborsofHamburgwithanyotherlocationwhichhaveamaximaldepthof2:

arangosh>varoptions={direction:'outbound',maxDepth:2,includeData:true};

arangosh>graph._countCommonNeighbors('germanCity/Hamburg',{},options,options);

_commonPropertiesGettheverticesofthegraphthatsharecommonproperties.

graph._commonProperties(vertex1Example,vertex2Examples,options)

Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertex1Exampleandvertex2Example.

ThecomplexityofthismethodisO(n)withnbeingthemaximalamountofverticesdefinedbytheparametersvertexExamples.

Parameters

vertex1Examples(optional)Filterthesetofsourcevertices,seeDefinitionofexamples

vertex2Examples(optional)Filterthesetofverticescomparedto,seeDefinitionofexamples

options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:vertex1CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbesearchedforsourcevertices.vertex2CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbesearchedforcomparevertices.ignoreProperties:Oneoralistofattributenamesofadocumentthatshouldbeignored.

Examples

Arouteplannerexample,alllocationswiththesameproperties:

arangosh>graph._commonProperties({},{});

showexecutionresultsArouteplannerexample,allcitieswhichsharesamepropertiesexceptforpopulation.

arangosh>graph._commonProperties({},{},{ignoreProperties:'population'});

_countCommonPropertiesGettheamountofverticesofthegraphthatsharecommonproperties.

graph._countCommonProperties(vertex1Example,vertex2Examples,options)

Similarto_commonPropertiesbutreturnscountinsteadoftheobjects.

Examples

Arouteplannerexample,alllocationswiththesameproperties:

GraphFunctions

arangosh>graph._countCommonProperties({},{});

showexecutionresultsArouteplannerexample,allGermancitieswhichsharesamepropertiesexceptforpopulation.

arangosh>graph._countCommonProperties({},{},{vertex1CollectionRestriction:

'germanCity',

........>vertex2CollectionRestriction:'germanCity',ignoreProperties:'population'});

_paths

The_pathsfunctionreturnsallpathsofagraph.

graph._paths(options)

Thisfunctiondeterminesallavailablepathsinagraph.

ThecomplexityofthismethodisO(n*n*m)withnbeingtheamountofverticesinthegraphandmtheaverageamountofconnectededges;

Parameters

options(optional)Anobjectcontainingoptions,seebelow:direction:Thedirectionoftheedges.Possiblevaluesareany,inboundandoutbound(default).followCycles(optional):Ifsettotruethequeryfollowscyclesinthegraph,defaultisfalse.minLength(optional):Definestheminimallengthapathmusthavetobereturned(defaultis0).maxLength(optional):Definesthemaximallengthapathmusthavetobereturned(defaultis10).

Examples

Returnallpathsofthegraph"social":

arangosh>varg=examples.loadGraph("social");

arangosh>g._paths();

showexecutionresultsReturnallinboundpathsofthegraph"social"withamaximallengthof1andaminimallengthof2:

arangosh>varg=examples.loadGraph("social");

arangosh>g._paths({direction:'inbound',minLength:1,maxLength:2});

_shortestPath

The_shortestPathfunctionreturnsallshortestpathsofagraph.

graph._shortestPath(startVertexExample,endVertexExample,options)

GraphFunctions

Thisfunctiondeterminesallshortestpathsinagraph.Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforstartandendvertex.Thelengthofapathisbydefaulttheamountofedgesfromonestartvertextoanendvertex.Theoptionweightallowstheusertodefineanedgeattributerepresentingthelength.

Parameters

startVertexExample(optional)AnexampleforthedesiredstartVertices(seeDefinitionofexamples).endVertexExample(optional)AnexampleforthedesiredendVertices(seeDefinitionofexamples).options(optional)Anobjectcontainingoptions,seebelow:

direction:Thedirectionoftheedgesasastring.Possiblevaluesareoutbound,inboundandany(default).edgeCollectionRestriction:Oneormultipleedgecollectionnames.Onlyedgesfromthesecollectionswillbeconsideredforthepath.startVertexCollectionRestriction:Oneormultiplevertexcollectionnames.Onlyverticesfromthesecollectionswillbeconsideredasstartvertexofapath.endVertexCollectionRestriction:Oneormultiplevertexcollectionnames.Onlyverticesfromthesecollectionswillbeconsideredasendvertexofapath.weight:Thenameoftheattributeoftheedgescontainingthelengthasastring.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedaslength.IfnodefaultissuppliedthedefaultwouldbepositiveInfinitysothepathcouldnotbecalculated.

Examples

Arouteplannerexample,shortestpathfromallgermantoallfrenchcities:

arangosh>g._shortestPath({},{},{weight:'distance',endVertexCollectionRestriction:

'frenchCity',

........>startVertexCollectionRestriction:'germanCity'});

showexecutionresultsArouteplannerexample,shortestpathfromHamburgandColognetoLyon:

arangosh>g._shortestPath([{_id:'germanCity/Cologne'},{_id:'germanCity/Munich'}],

'frenchCity/Lyon',

........>{weight:'distance'});

_distanceTo

The_distanceTofunctionreturnsallpathsandtheredistancewithinagraph.

graph._distanceTo(startVertexExample,endVertexExample,options)

Thisfunctionisawrapperofgraph._shortestPath.Itdoesnotreturntheactualpathbutonlythedistancebetweentwovertices.

Examples

Arouteplannerexample,shortestdistancefromallgermantoallfrenchcities:

arangosh>g._distanceTo({},{},{weight:'distance',endVertexCollectionRestriction:

'frenchCity',

........>startVertexCollectionRestriction:'germanCity'});

GraphFunctions

showexecutionresultsArouteplannerexample,shortestdistancefromHamburgandColognetoLyon:

arangosh>g._distanceTo([{_id:'germanCity/Cologne'},{_id:'germanCity/Munich'}],

'frenchCity/Lyon',

........>{weight:'distance'});

_absoluteEccentricityGettheeccentricityoftheverticesdefinedbytheexamples.

graph._absoluteEccentricity(vertexExample,options)

Parameters

vertexExample(optional)Filterthevertices,seeDefinitionofexamplesoptions(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:

direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).edgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.startVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredforsourcevertices.endVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredfortargetvertices.weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheeccentricitycannotbecalculated.

Examples

Arouteplannerexample,theabsoluteeccentricityofalllocations.

arangosh>graph._absoluteEccentricity({});

showexecutionresultsArouteplannerexample,theabsoluteeccentricityofalllocations.Thisconsiderstheactualdistances.

arangosh>graph._absoluteEccentricity({},{weight:'distance'});

showexecutionresultsArouteplannerexample,theabsoluteeccentricityofallcitiesregardingonlyoutboundpaths.

arangosh>graph._absoluteEccentricity({},{startVertexCollectionRestriction:

'germanCity',

........>direction:'outbound',weight:'distance'});

GraphFunctions

_eccentricity

Getthenormalizedeccentricityoftheverticesdefinedbytheexamples.

graph._eccentricity(vertexExample,options)

Similarto_absoluteEccentricitybutreturnsanormalizedresult.

Examples

Arouteplannerexample,theeccentricityofalllocations.

arangosh>graph._eccentricity();

showexecutionresultsArouteplannerexample,theweightedeccentricity.

arangosh>graph._eccentricity({weight:'distance'});

_absoluteCloseness

Gettheclosenessoftheverticesdefinedbytheexamples.

graph._absoluteCloseness(vertexExample,options)

Parameters

direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).edgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.startVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredforsourcevertices.endVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredfortargetvertices.weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheclosenesscannotbecalculated.

Examples

Arouteplannerexample,theabsoluteclosenessofalllocations.

arangosh>graph._absoluteCloseness({});

showexecutionresultsArouteplannerexample,theabsoluteclosenessofalllocations.Thisconsiderstheactualdistances.

GraphFunctions

arangosh>graph._absoluteCloseness({},{weight:'distance'});

showexecutionresultsArouteplannerexample,theabsoluteclosenessofallGermanCitiesregardingonlyoutboundpaths.

arangosh>graph._absoluteCloseness({},{startVertexCollectionRestriction:'germanCity',

........>direction:'outbound',weight:'distance'});

_closeness

Getthenormalizedclosenessofgraphsvertices.

graph._closeness(options)

Similarto_absoluteClosenessbutreturnsanormalizedvalue.

Examples

Arouteplannerexample,thenormalizedclosenessofalllocations.

arangosh>graph._closeness();

showexecutionresultsArouteplannerexample,theclosenessofalllocations.Thisconsiderstheactualdistances.

arangosh>graph._closeness({weight:'distance'});

showexecutionresultsArouteplannerexample,theclosenessofallcitiesregardingonlyoutboundpaths.

arangosh>graph._closeness({direction:'outbound',weight:'distance'});

_absoluteBetweennessGetthebetweennessofallverticesinthegraph.

graph._absoluteBetweenness(vertexExample,options)

Parameters

direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthis

GraphFunctions

defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencethebetweenesscannotbecalculated.

Examples

Arouteplannerexample,theabsolutebetweennessofalllocations.

arangosh>graph._absoluteBetweenness({});

showexecutionresultsArouteplannerexample,theabsolutebetweennessofalllocations.Thisconsiderstheactualdistances.

arangosh>graph._absoluteBetweenness({weight:'distance'});

Arouteplannerexample,theabsolutebetweennessofallcitiesregardingonlyoutboundpaths.

arangosh>graph._absoluteBetweenness({direction:'outbound',weight:'distance'});

_betweenness

Getthenormalizedbetweennessofgraphsvertices.

graph_module._betweenness(options)

Similarto_absoluteBetweenessbutreturnsnormalizedvalues.

Examples

Arouteplannerexample,thebetweennessofalllocations.

arangosh>graph._betweenness();

showexecutionresultsArouteplannerexample,thebetweennessofalllocations.Thisconsiderstheactualdistances.

arangosh>graph._betweenness({weight:'distance'});

showexecutionresultsArouteplannerexample,thebetweennessofallcitiesregardingonlyoutboundpaths.

arangosh>graph._betweenness({direction:'outbound',weight:'distance'});

GraphFunctions

_radiusGettheradiusofagraph.

Parameters

options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheradiuscannotbecalculated.

Examples

Arouteplannerexample,theradiusofthegraph.

arangosh>graph._radius();

Arouteplannerexample,theradiusofthegraph.Thisconsiderstheactualdistances.

arangosh>graph._radius({weight:'distance'});

Arouteplannerexample,theradiusofthegraphregardingonlyoutboundpaths.

arangosh>graph._radius({direction:'outbound',weight:'distance'});

_diameter

Getthediameterofagraph.

graph._diameter(graphName,options)

Parameters

options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheradiuscannotbecalculated.

Examples

GraphFunctions

Arouteplannerexample,thediameterofthegraph.

arangosh>graph._diameter();

Arouteplannerexample,thediameterofthegraph.Thisconsiderstheactualdistances.

arangosh>graph._diameter({weight:'distance'});

Arouteplannerexample,thediameterofthegraphregardingonlyoutboundpaths.

arangosh>graph._diameter({direction:'outbound',weight:'distance'});

GraphFunctions

SmartGraphsThisfeatureisonlyavailableintheEnterpriseEdition.

Thischapterdescribesthesmart-graphmodule.Itenablesyoutomanagegraphsatscale,itwillgiveavastperformancebenefitforallgraphsshardedinanArangoDBCluster.Onasingleserverthisfeatureispointless,henceitisonlyavailableinaclustermode.IntermsofqueryingthereisnodifferencebetweensmartandGeneralGraphs.Theformerareatransparentreplacementforthelatter.SoforqueryingthegraphpleaserefertoAQLGraphOperationsandGraphFunctionssections.TheoptimizeriscleverenoughtoidentifyifweareonaSmartGraphornot.

Thedifferenceisonlyinthemanagementsection:creatingandmodifyingtheunderlyingcollectionsofthegraph.ForadetailedAPIreferencepleaserefertoSmartGraphManagement.

Whatmakesagraphsmart?

Mostgraphshaveonefeaturethatdividestheentiregraphintoseveralsmallersubgraphs.Thesesubgraphshavealargeamountofedgesthatonlyconnectverticesinthesamesubgraphandonlyhavefewedgesconnectingverticesfromothersubgraphs.Examplesforthesegraphsare:

SocialNetworks

Typicallythefeaturehereistheregion/countryuserslivein.Everyusertypicalliyhasmorecontactsinthesameregion/countrythenshehasinotherregions/countries

TransportSystems

Forthosealsothefeatureistheregion/country.Youhavemanylocaltransportionbutonlyfewaccrosscountries.

E-Commerce

Inthiscaseprobablythecategoryofproductsisagoodfeature.Oftenproductsofthesamecategoryareboughttogether.

Ifthisfeatureisknown,SmartGraphscanmakeuseifit.WhencreatingaSmartGraphyouhavetodefineasmartAttribute,whichisthenameofanattributestoredineveryvertex.Thegraphwillthanbeautomaticallyshardedinsuchawaythatallverticeswiththesamevaluearestoredonthesamephysicalmachine,alledgesconnectingverticeswithidenticalsmartAttributevaluesarestoredonthismachineaswell.Duringquerytimethequeryoptimizerandthequeryexecutorbothknowforeverydocumentexactlywhereitisstoredandcantherebyminimizenetworkoverhead.Everythingthatcanbecomputedlocallywillbecomputedlocally.

BenefitsofSmartGraphs

Becauseoftheabovedescribedguaranteedsharding,theperformanceofqueriesthatonlycoveronesubgraphhaveaperformancealmostequaltoanonlylocalcomputation.Queriesthatcovermorethanonesubgraphrequiresomenetworkoverhead.Themoresubgraphsaretouchedthemorenetworkcostwillapply.HowevertheoverallperformanceisneverworsethanthesamequeryonaGeneralGraph.

Gettingstarted

FirstofallSmartGraphscannotuseexistingcollections,whenswitchingtoSmartGraphfromanexistingdatasetyouhavetoimportthedataintoafreshSmartGraph.Thisswitchcanbeeasilyachievedwitharangodumpandarangorestore.TheonlythingyouhavetochangeinthispipelineisthatyoucreatethenewcollectionswiththeSmartGraphbeforestartingarangorestore.

Createagraph

IncomparisontoGeneralGraphwehavetoaddmoreoptionswhencreatingthegraph.ThetwooptionssmartGraphAttributeandnumberOfShardsarerequiredandcannotbemodifedlater.

arangosh>vargraph_module=require("@arangodb/smart-graph");

arangosh>vargraph=graph_module._create("myGraph",[],[],{smartGraphAttribute:"region",numberOfShards:9});

arangosh>graph;

[SmartGraphmyGraphEdgeDefinitions:[]VertexCollections:[]]

Addsomevertexcollections

SmartGraphs

ThisisagainidenticaltoGeneralGraph.Themodulewillsetupcorrectshardingforallthesecollections.Note:Thecollectionshavetobenew.

arangosh>graph._addVertexCollection("shop");

arangosh>graph._addVertexCollection("customer");

arangosh>graph._addVertexCollection("pet");

arangosh>graph;

[SmartGraphmyGraphEdgeDefinitions:[]VertexCollections:["shop","customer","pet"]]

DefinerelationsontheGraph

arangosh>varrel=graph_module._relation("isCustomer",["shop"],["customer"]);

arangosh>graph._extendEdgeDefinitions(rel);

arangosh>graph;

[SmartGraphmyGraphEdgeDefinitions:["isCustomer:[shop]->[customer]"]VertexCollections:["pet"]]

SmartGraphs

SmartGraphManagementThischapterdescribestheJavaScriptinterfaceforcreatingandmodifyingSmartGraphs.AtfirstyouhavetonotethateverySmartGraphisaspecializedversionofaGeneralGraph,whichmeansalloftheGeneralGraphfunctionalityisavailableonaSmartGraphaswell.Themajordifferenceofbothmodulesishandlingoftheunderlyingcollections,theGeneralGraphdoesnotenforceormaintainanyshardingofthecollectionsandcanthereforcombinearbitrarysetsofexistingcollections.SmartGraphsenforceandrelyonaspecialshardingoftheunderlyingcollectionsandhencecanonlyworkwithcollectionsthatarecreatedthroughtheSmartGraphitself.ThisalsomeansthatSmartGraphscannotbeoverlapping,acollectioncaneitherbeshardedforoneSmartGraphorfortheother.IfyouneedtomakesurethatallqueriescanbeexecutedwithSmartGraphperformance,justcreateonelargeSmartGraphcoveringeverythingandqueryitstatingthesubsetofedgecollectionsexplicitly.TogenerallyunderstandtheconceptofthismodulepleasereadthechapteraboutGeneralGraphManagementfirst.Inthefollowingwewillonlydescribetheoverloadedfunctionality.Everythingelseworksidenticalinbothmodules.

Createagraph

AlsoSmartGraphsrequireedgerelationstobecreated,theformatoftherelationsisidentical.TheonlydifferenceisthatallcollectionsusedwithintherelationstocreateanewSmartGraphcannotexistyet.TheyhavetobecreatedbytheGraphinordertoenforcethecorrectsharding.

Createagraph

graph_module._create(graphName,edgeDefinitions,orphanCollections,smartOptions)

ThecreationofagraphrequiresthenameandsomeSmartGraphoptions.DuetotheAPIedgeDefinitionsandorphanCollectionshavetobegiven,butbothcanbeemptyarraysandcanbecreatedlater.TheedgeDefinitionscanbecreatedusingtheconveniencemethod_relationknownfromthegeneral-graphmodule,whichisalsoavailablehere.orphanCollectionsagainisjustalistofadditionalvertexcollectionswhicharenotyetconnectedviaedgesbutshouldfollowthesameshardingtobeconnectedlateron.Allcollectionsusedwithinthecreationprocessarenewlycreated.Theprocesswillfailifoneofthemalreadyexists.Allnewlycreatedcollectionswillimmediatelybedroppedagaininthefailedcase.

Parameters

graphName(required)UniqueidentifierofthegraphedgeDefinitions(required)Listofrelationdefinitionobjects,maybeemptyorphanCollections(required)Listofadditionalvertexcollectionnames,maybeemptysmartOptions(required)AJSONobjecthavingthefollowingkeys:

numberOfShards(required)Thenumberofshardsthatwillbecreatedforeachcollection.Tomaintainthecorrectshardingallcollectionsneedanidenticalnumberofshards.Thiscannotbemodifiedaftercreationofthegraph.smartGraphAttribute(required)Theattributethatwillbeusedforsharding.Allverticesarerequiredtohavethisattributesetandithastobeastring.Edgesderivetheattributefromtheirconnectedvertices.

Examples

Createanemptygraph,edgedefinitionscanbeaddedatruntime:

arangosh>vargraph=graph_module._create("myGraph",[],[],{smartGraphAttribute:"region",numberOfShards:9});

[SmartGraphmyGraphEdgeDefinitions:[]VertexCollections:[]]

Createagraphusinganedgecollectionedgesandasinglevertexcollectionvertices

arangosh>varedgeDefinitions=[graph_module._relation("edges","vertices","vertices")];

arangosh>vargraph=graph_module._create("myGraph",edgeDefinitions,[],{smartGraphAttribute:"region",numberOfShards:9});

[SmartGraphmyGraphEdgeDefinitions:["edges:[vertices]->[vertices]"]VertexCollections:[]]

Createagraphwithedgedefinitionsandorphancollections:

arangosh>varedgeDefinitions=[graph_module._relation("myRelation",["male","female"],["male","female"])];

arangosh>vargraph=graph_module._create("myGraph",edgeDefinitions,["sessions"],{smartGraphAttribute:"region",numberOfSh

ards:9});

[GraphmyGraphEdgeDefinitions:[

"myRelation:[female,male]->[female,male]"

]VertexCollections:[

"sessions"

ModifyagraphdefinitionduringruntimeAfteryouhavecreatedaSmartGraphitsdefinitionisalsonotimmutable.Youcanstilladdorremoverelations.ThisisagainidenticaltoGeneralGraphs.Howeverthereisoneimportantdifference:Youcanonlyaddcollectionsthateitherdonotexist,orthathavebeencreatedbythisgraphearlier.Thelatercanbeachievedifyouforexampleremoveanorphancollectionfromthisgraph,withoutdroppingthecollectionitself.Thanaftersometimeyoudecidetoadditagain,itcanbeused.Thisisbecausetheenforcedshardingisstillappliedtothisvertexcollection,henceitissuitabletobeaddedagain.

Removeavertexcollection

Removeavertexcollectionfromthegraph

graph._removeVertexCollection(vertexCollectionName,dropCollection)

InmostcasesthisfunctionworksidenticallytotheGeneralGraphone.Butthereisonespecialcase:Thefirstvertexcollectionaddedtothegraph(eitherorphanorwithinarelation)definestheshardingforallcollectionswithinthegraph.Thiscollectioncanneverberemovedfromthegraph.

Parameters

vertexCollectionName(required)Nameofvertexcollection.dropCollection(optional)Iftruethecollectionwillbedroppedifitisnotusedinanyothergraph.Default:false.

Examples

Thefollowingexampleshowsthatyoucannotdroptheinitialcollection.Youhavetodropthecompletegraph.Ifyoujustwanttogetridofthedatatruncateit.

arangosh>vargraph_module=require("@arangodb/smart-graph")

arangosh>varrelation=graph_module._relation("edges","vertices","vertices");

arangosh>vargraph=graph_module._create("myGraph",[relation],["other"],{smartGraphAttribute:"region",numberOfShards:

"other"

arangosh>graph._deleteEdgeDefinition("edges");

"vertices",

"other"

arangosh>graph._removeVertexCollection("other");

"vertices"

arangosh>graph._removeVertexCollection("vertices");

ArangoError4002:cannotdropthissmartcollection

TraversalsArangoDBprovidesseveralwaystoquerygraphdata.Verysimpleoperationscanbecomposedwiththelow-leveledgemethodsedges,inEdges,andoutEdgesforedgecollections.Theseworkonnamedandanonymousgraphs.Formorecomplexoperations,ArangoDBprovidespredefinedtraversalobjects.

AlsoTraversalshavebeenaddedtoAQL.PleasereadthechapteraboutAQLtraversersalsbeforeyoucontinuereadinghere.MostofthetraversalcasesarecoveredbyAQLandwillbeexecutedinanoptimizedway.OnlyifthelogicforyouristoocomplextobedefinedusingAQLfiltersyoucanusethetraversalobjectdefinedherewhichgivesyoucompleteprogrammaticaccesstothedata.

Foranyofthefollowingexamples,we'llbeusingtheexamplecollectionsvande,populatedwithcontinents,countriesandcapitalsdatalistedbelow(seeExampleData).

StartingfromScratchArangoDBprovidestheedges,inEdges,andoutEdgesmethodsforedgecollections.Thesemethodscanbeusedtoquicklydetermineifavertexisconnectedtoothervertices,andwhich.ThisfunctionalitycanbeexploitedtowriteverysimplegraphqueriesinJavaScript.

Forexample,todeterminewhichedgesarelinkedtotheworldvertex,wecanuseinEdges:

db.e.inEdges('v/world').forEach(function(edge){

require("@arangodb").print(edge._from,"->",edge.type,"->",edge._to);

inEdgeswillgiveusallingoingedgesforthespecifiedvertexv/world.TheresultisaJavaScriptarraythatwecaniterateoverandprinttheresults:

v/continent-africa->is-in->v/world

v/continent-south-america->is-in->v/world

v/continent-asia->is-in->v/world

v/continent-australia->is-in->v/world

v/continent-europe->is-in->v/world

v/continent-north-america->is-in->v/world

Note:edges,inEdges,andoutEdgesreturnanarrayofedges.Ifwewanttoretrievethelinkedvertices,wecanuseeachedges'_fromand_toattributesasfollows:

db.e.inEdges('v/world').forEach(function(edge){

require("@arangodb").print(db._document(edge._from).name,"->",edge.type,"->",db._document(edge._to).name);

Weareusingthedocumentmethodfromthedbobjecttoretrievetheconnectedverticesnow.

Whilethismaybesufficientforone-levelgraphoperations,writingatraversalbyhandmaybecometoocomplexformulti-leveltraversals.

Traversals

GettingstartedTouseatraversalobject,wefirstneedtorequirethetraversalmodule:

vartraversal=require("@arangodb/graph/traversal");

varexamples=require("@arangodb/graph-examples/example-graph.js");

examples.loadGraph("worldCountry");

Wethenneedtosetupaconfigurationforthetraversalanddetermineatwhichvertextostartthetraversal:

varconfig={

datasource:traversal.generalGraphDatasourceFactory("worldCountry"),

strategy:"depthfirst",

order:"preorder",

filter:traversal.visitAllFilter,

expander:traversal.inboundExpander,

maxDepth:1

varstartVertex=db._document("v/world");

Note:ThestartVertexneedstobeadocument,notonlyadocumentid.

Wecanthencreateatraverserandstartthetraversalbycallingitstraversemethod.Notethattraverseneedsaresultobject,whichitcanmodifyinplace:

varresult={

visited:{

vertices:[],

paths:[]

vartraverser=newtraversal.Traverser(config);

traverser.traverse(result,startVertex);

Finally,wecanprintthecontentsoftheresultsobject,limitedtothevisitedvertices.Wewillonlyprintthenameandtypeofeachvisitedvertexforbrevity:

require("@arangodb").print(result.visited.vertices.map(function(vertex){

returnvertex.name+"("+vertex.type+")";

Thefullscript,whichincludesallstepscarriedoutsofaristhus:

varconfig={

datasource:traversal.generalGraphDatasourceFactory("worldCountry"),

order:"preorder",

maxDepth:1

varstartVertex=db._document("v/world");

varresult={

visited:{

vertices:[],

paths:[]

Theresultisanarrayofverticesthatwerevisitedduringthetraversal,startingatthestartvertex(i.e.v/worldinourexample):

"World(root)",

"Africa(continent)",

"Asia(continent)",

"Australia(continent)",

"Europe(continent)",

"NorthAmerica(continent)",

"SouthAmerica(continent)"

Note:Theresultislimitedtoverticesdirectlyconnectedtothestartvertex.WeachievedthisbysettingthemaxDepthattributeto1.Notsettingitwouldreturnthefullarrayofvertices.

TraversalDirection

Fortheexamplescontainedinthismanual,we'llbestartingthetraversalsatvertexv/world.Verticesinourgraphareconnectedlikethis:

v/world<-is-in<-continent(Africa)<-is-in<-country(Algeria)<-is-in<-capital(Algiers)

Togetanymeaningfulresults,wemusttraversethegraphininboundorder.Thismeans,we'llbefollowingallincomingedgesoftoavertex.Inthetraversalconfiguration,wehavespecifiedthisviatheexpanderattribute:

varconfig={

expander:traversal.inboundExpander

Forothergraphs,wemightwanttotraverseviatheoutgoingedges.Forthis,wecanusetheoutboundExpander.ThereisalsoananyExpander,whichwillfollowbothoutgoingandincomingedges.Thisshouldbeusedwithcareandthetraversalshouldalwaysbelimitedtoamaximumnumberofiterations(e.g.usingthemaxIterationsattribute)inordertoterminateatsomepoint.

Toinvokethedefaultoutboundexpanderforagraph,simplyusethepredefinedfunction:

varconfig={

expander:traversal.outboundExpander

Pleasenotetheoutboundexpanderwillnotproduceanyoutputfortheexamplesifwestillstartthetraversalatthev/worldvertex.

Still,wecanusetheoutboundexpanderifwestartsomewhereelseinthegraph,e.g.

varconfig={

datasource:traversal.generalGraphDatasourceFactory("world_graph"),

order:"preorder",

expander:traversal.outboundExpander

varstartVertex=db._document("v/capital-algiers");

varresult={

visited:{

vertices:[],

paths:[]

Theresultis:

"Algiers(capital)",

"Algeria(country)",

"World(root)"

whichconfirmsthatnowwe'regoingoutbound.

TraversalStrategy

Depth-firsttraversals

Thevisitationorderofverticesisdeterminedbythestrategyandorderattributessetintheconfiguration.Wechosedepthfirstandpreorder,meaningthetraverserwillvisiteachvertexbeforehandlingconnectededges(pre-order),anddescendintoanyconnectededgesbeforeprocessingotherverticesonthesamelevel(depth-first).

Let'sremovethemaxDepthattributenow.We'llnowbegettingallvertices(directlyandindirectlyconnectedtothestartvertex):

varconfig={

order:"preorder",

expander:traversal.inboundExpander

varresult={

visited:{

vertices:[],

paths:[]

Theresultwillbealongerarray,assembledindepth-first,pre-orderorder.Foreachcontinentfound,thetraverserwilldescendintolinkedcountries,andthenintothelinkedcapital:

"World(root)",

"Algeria(country)",

"Algiers(capital)",

"Angola(country)",

"Luanda(capital)",

"Botswana(country)",

"Gaborone(capital)",

"BurkinaFaso(country)",

"Ouagadougou(capital)",

Let'sswitchtheorderattributefrompreordertopostorder.Thiswillmakethetraverservisitverticesafterallconnectedverticeswerevisited(i.e.mostdistantverticeswillbeemittedfirst):

"Algiers(capital)",

"Algeria(country)",

"Luanda(capital)",

"Angola(country)",

"Gaborone(capital)",

"Botswana(country)",

"Ouagadougou(capital)",

"Bujumbura(capital)",

"Burundi(country)",

"Yaounde(capital)",

"Cameroon(country)",

"N'Djamena(capital)",

"Chad(country)",

"Yamoussoukro(capital)",

"Coted'Ivoire(country)",

"Cairo(capital)",

"Egypt(country)",

"Asmara(capital)",

"Eritrea(country)",

Breadth-firsttraversals

Ifwegobacktopreorder,butchangethestrategytobreadth-firstandre-runthetraversal,we'llseethatthereturnorderchanges,anditemsonthesamelevelwillbereturnedadjacently:

"World(root)",

"Asia(continent)",

"SouthAmerica(continent)",

"Burundi(country)",

"Cameroon(country)",

"Chad(country)",

"Algeria(country)",

"Angola(country)",

Note:Theorderofitemsreturnedforthesamelevelisundefined.Thisisbecausethereisnonaturalorderofedgesforavertexwithmultipleconnectededges.Toexplicitlysettheorderforedgesonthesamelevel,youcanspecifyanedgecomparatorfunctionwiththesortattribute:

varconfig={

sort:function(l,r){returnl._key<r._key?1:-1;}

Theargumentslandrareedgedocuments.Thiswilltraverseedgesofthesamevertexinbackward_keyorder:

"World(root)",

"SouthAmerica(continent)",

"Asia(continent)",

"Ecuador(country)",

"Colombia(country)",

"Chile(country)",

"Brazil(country)",

"Bolivia(country)",

"Argentina(country)",

Note:Thisattributeonlyworksfortheusualexpanderstraversal.inboundExpander,traversal.outboundExpander,traversal.anyExpanderandtheircorresponding"WithLabels"variants.Ifyouareusingcustomexpandersyouhavetoorganizethesortingwithinthespecifiedexpander.

WritingCustomVisitors

Sofarwehaveusedmuchofthetraverser'sdefaultfunctions.Thetraverserisveryconfigurableandmanyofthedefaultfunctionscanbeoverriddenwithcustomfunctionality.

Forexample,wehavebeenusingthedefaultvisitorfunction(whichisalwaysusediftheconfigurationdoesnotcontainthevisitorattribute).Thedefaultvisitorfunctioniscalledforeachvertexinatraversal,andwillpushitintotheresult.Thisisthereasonwhytheresultvariablelookeddifferentafterthetraversal,andneededtobeinitializedbeforethetraversalwasstarted.

Notethatthedefaultvisitor(namedtrackingVisitor)willaddeveryvisitedvertexintotheresult,includingthefullpathsfromthestartvertex.Thisisusefulforlearninganddebuggingpurposes,butshouldbeavoidedinproductionbecauseitmightproduce(andcopy)hugeamountsofdata.Instead,onlythosedatashouldbecopiedintotheresultthatareactuallynecessary.

Thetraversercomeswiththefollowingpredefinedvisitors:

trackingVisitor:thisisthedefaultvisitor.Itwillcopyalldataofeachvisitedvertexplusthefullpathinformationintotheresult.Thiscanbeslowiftheresultsetishugeorverticescontainalotofdata.countingVisitor:thisisaverylightweightvisitor:allitdoesisincreaseacounterintheresultforeachvertexvisited.Vertexdataandpathswillnotbecopiedintotheresult.doNothingVisitor:ifnoactionshallbecarriedoutwhenavertexisvisited,thisvisitorcanbeemployed.Itwillnotdoanythingandwillthusbefast.Itcanbeusedforperformancecomparisonswithothervisitors.

Wecanalsowriteourownvisitorfunctionifwewantto.Thegeneralfunctionsignatureforvisitorfunctionsisasfollows:

varconfig={

visitor:function(config,result,vertex,path,connected){...}

Note:theconnectedparametervaluewillonlybesetifthetraversalorderissettopreorder-expander.Otherwise,thisparameterwon'tbesetbythetraverser.

Visitorfunctionsarenotexpectedtoreturnanyvalues.Instead,theycanmodifytheresultvariable(e.g.bypushingthecurrentvertexintoit),ordoanythingelse.Forexample,wecancreateasimplevisitorfunctionthatonlyprintsinformationaboutthecurrentvertexaswetraverse:

varconfig={

order:"preorder",

visitor:function(config,result,vertex,path){

require("@arangodb").print("visitingvertex",vertex.name);

traverser.traverse(undefined,startVertex);

Towriteavisitorthatincrementsacountereachtimeavertexisvisited,wecouldwritethefollowingcustomvisitor:

config.visitor=function(config,result,vertex,path,connected){

if(!result){

result={};

if(!result.hasOwnProperty('count')){

result.count=0;

++result.count;

Notethatsuchvisitorisalreadypredefined(it'sthecountingVisitordescribedabove).Itcanbeusedasfollows:

config.visitor=traversal.countingVisitor;

Anotherexampleofavisitorisonethatcollectsthe_idvaluesofallverticesvisited:

if(!result){

result={};

if(!result.hasOwnProperty("visited")){

result.visited={vertices:[]};

result.visited.vertices.push(vertex._id);

Whenthetraversalorderissettopreorder-expander,thetraverserwillpassafifthparametervalueintothevisitorfunction.Thisparametercontainstheconnectededgesofthevisitedvertexasanarray.Thiscanbehandybecauseinthiscasethevisitorwillgetallinformationaboutthevertexandtheconnectededgestogether.

Forexample,thefollowingvisitorcanbeusedtoprintonlyleafnodes(thatdonothaveanyfurtherconnectededges):

if(connected&&connected.length===0){

require("@arangodb").print("foundaleaf-node:",vertex);

Notethatforthisvisitortowork,thetraversalorderattributeneedstobesettothevaluepreorder-expander.

FilteringVerticesandEdges

FilteringVertices

Sofarwehaveprintedorreturnedallverticesthatwerevisitedduringthetraversal.Thisisnotalwaysrequired.Iftheresultshallberestricttojustspecificvertices,wecanuseafilterfunctionforvertices.Itcanbedefinedbysettingthefilterattributeofatraversalconfiguration,e.g.:

varconfig={

filter:function(config,vertex,path){

if(vertex.type!=='capital'){

return'exclude';

Theabovefilterfunctionwillexcludeallverticesthatdonothaveatypevalueofcapital.Thefilterfunctionwillbecalledforeachvertexfoundduringthetraversal.Itwillreceivethetraversalconfiguration,thecurrentvertex,andthefullpathfromthetraversalstartvertextothecurrentvertex.Thepathconsistsofanarrayofedges,andanarrayofvertices.Wecouldalsofiltereverythingbutcapitalsby

checkingthelengthofthepathfromthestartvertextothecurrentvertex.Capitalswillhaveadistanceof3fromthev/worldstartvertex(capital→is-in→country→is-in→continent→is-in→world):

varconfig={

if(path.edges.length<3){

return'exclude';

Note:Ifafilterfunctionreturnsnothing(orundefined),thecurrentvertexwillbeincluded,andallconnectededgeswillbefollowed.Ifafilterfunctionreturnsexcludethecurrentvertexwillbeexcludedfromtheresult,andallstillallconnectededgeswillbefollowed.Ifafilterfunctionreturnsprune,thecurrentvertexwillbeincluded,butnoconnectededgeswillbefollowed.

Forexample,thefollowingfilterfunctionwillnotdescendintoconnectededgesofcontinents,limitingthedepthofthetraversal.Still,continentverticeswillbeincludedintheresult:

varconfig={

if(vertex.type==='continent'){

return'prune';

Itisalsopossibletocombineexcludeandprunebyreturninganarraywithbothvalues:

return['exclude','prune'];

FilteringEdges

Itispossibletoexcludecertainedgesfromthetraversal.Tofilteronedges,afilterfunctioncanbedefinedviatheexpandFilterattribute.TheexpandFilterisafunctionwhichiscalledforeachedgeduringatraversal.

Itwillreceivethecurrentedge(edgevariable)andthevertexwhichtheedgeconnectsto(inthedirectionofthetraversal).Italsoreceivesthecurrentpathfromthestartvertexuptothecurrentvertex(excludingthecurrentedgeandthevertextheedgepointsto).

Ifthefunctionreturnstrue,theedgewillbefollowed.Ifthefunctionreturnsfalse,theedgewillnotbefollowed.Hereisaverysimplecustomedgefilterfunctionimplementation,whichsimplyincludesedgesifthe(edges)pathlengthislessthan1,andwillexcludeanyotheredges.Thiswilleffectivelyterminatethetraversalafterthefirstlevelofedges:

varconfig={

expandFilter:function(config,vertex,edge,path){

return(path.edges.length<1);

WritingCustomExpanders

Theedgesconnectedtoavertexaredeterminedbytheexpander.Sofarwehaveusedadefaultexpander(thedefaultinboundexpandertobeprecise).Thedefaultinboundexpandersimplyenumeratesallconnectedingoingedgesforavertex,basedontheedgecollectionspecifiedinthetraversalconfiguration.

Thereisalsoadefaultoutboundexpander,whichwillenumerateallconnectedoutgoingedges.Finally,thereisananyexpander,whichwillfollowbothingoingandoutgoingedges.

Ifconnectededgesmustbedeterminedinsomedifferentfashionforwhateverreason,acustomexpandercanbewrittenandregisteredbysettingtheexpanderattributeoftheconfiguration.Theexpanderfunctionsignatureisasfollows:

varconfig={

expander:function(config,vertex,path){...}

Itistheexpander'sresponsibilitytoreturnalledgesandverticesdirectlyconnectedtothecurrentvertex(whichispassedviathevertexvariable).Thefullpathfromthestartvertexuptothecurrentvertexisalsosuppliedviathepathvariable.Anexpanderisexpectedtoreturnanarrayofobjects,whichneedtohaveanedgeandavertexattributeeach.

Note:Ifyouwanttorelyonaparticularorderinwhichtheedgesaretraversed,youhavetosorttheedgesreturnedbyyourexpanderwithinthecodeoftheexpander.Thefunctionstogetoutbound,inboundoranyedgesfromavertexdonotguaranteeanyparticularorder!

Acustomimplementationofaninboundexpandercouldlooklikethis(thisisanon-deterministicexpander,whichrandomlydecideswhetherornottoincludeconnectededges):

varconfig={

expander:function(config,vertex,path){

varconnected=[];

vardatasource=config.datasource;

datasource.getInEdges(vertex._id).forEach(function(edge){

if(Math.random()>=0.5){

connected.push({edge:edge,vertex:(edge._from)});

returnconnected;

Acustomexpandercanalsobeusedasanedgefilterbecauseithasfullcontroloverwhichedgeswillbereturned.

Followingaretwoexamplesofcustomexpandersthatpickedgesbasedonattributesoftheedgesandtheconnectedvertices.

Findingtheconnectededges/verticesbasedonanattributewhenintheconnectedvertices.Thegoalistofollowtheedgethatleadstothevertexwiththehighestvalueinthewhenattribute:

varconfig={

//determinealloutgoingedges

varoutEdges=datasource.getOutEdges(vertex);

if(outEdges.length===0){

return[];

vardata=[];

outEdges.forEach(function(edge){

data.push({edge:edge,vertex:datasource.getInVertex(edge)});

//sortoutgoingverticesaccordingto"when"attributevalue

data.sort(function(l,r){

if(l.vertex.when===r.vertex.when){

return0;

return(l.vertex.when<r.vertex.when?1:-1);

//pickfirstvertexfound(withhighest"when"attributevalue)

return[data[0]];

Findingtheconnectededges/verticesbasedonanattributewhenintheedgeitself.Thegoalistopicktheoneedge(outofpotentiallymany)thathasthehighestwhenattributevalue:

varconfig={

//determinealloutgoingedges

varoutEdges=datasource.getOutEdges(vertex);

if(outEdges.length===0){

return[];//returnanemptyarray

//sortalloutgoingedgesaccordingto"when"attribute

outEdges.sort(function(l,r){

if(l.when===r.when){

return0;

return(l.when<r.when?-1:1);

//returnfirstedge(theonewithhighest"when"value)

varedge=outEdges[0];

varv=datasource.getInVertex(edge);

return[{edge:edge,vertex:v}];

catch(e){}

return[];

HandlingUniqueness

Graphsmaycontaincycles.Tobeontopofwhathappenswhenatraversalencountersavertexoranedgeithasalreadyvisited,thereareconfigurationoptions.

Thedefaultconfigurationistovisiteveryvertex,regardlessofwhetheritwasalreadyvisitedinthesametraversal.However,edgeswillbydefaultonlybefollowediftheyarenotalreadypresentinthecurrentpath.

Imaginethefollowinggraphwhichcontainsacycle:

A->B->C->A

WhenthetraversalfindstheedgefromCtoA,itwillbydefaultfollowit.Thisisbecausewehavenotseenthisedgeyet.ItwillalsovisitvertexAagain.Thisisbecausebydefaultallverticeswillbevisited,regardlessofwhetheralreadyvisitedornot.

However,thetraversalwillnotagainfollowingtheoutgoingedgefromAtoB.ThisisbecausewealreadyhavetheedgefromAtoBinourcurrentpath.

Thesedefaultsettingswillpreventinfinitetraversals.

Toadjusttheuniquenessforvisitingvertices,therearethefollowingoptionsforuniqueness.vertices:

"none":alwaysvisitavertices,regardlessofwhetheritwasalreadyvisitedornot"global":visitavertexonlyifitwasnotvisitedinthetraversal"path":visitavertexifitisnotincludedinthecurrentpath

Toadjusttheuniquenessforfollowingedges,therearethefollowingoptionsforuniqueness.edges:

"none":alwaysfollowanedge,regardlessofwhetheritwasfollowedbefore"global":followanedgeonlyifitwasn'tfollowedinthetraversal"path":followanedgeifitisnotincludedinthecurrentpath

Notethatuniquenesscheckingwillhavesomeeffectonbothruntimeandmemoryusage.Forexample,whenuniquenesschecksaresetto"global" ,arraysofvisitedverticesandedgesmustbekeptinmemorywhilethetraversalisexecuted.Globaluniquenessshouldthusonlybeusedwhenatraversalisexpectedtovisitfewnodes.

Intermsofruntime,turningoffuniquenesschecks(bysettingbothoptionsto"none")isthebestchoice,butitisonlysafeforgraphsthatdonotcontaincycles.Whenuniquenesschecksaredeactivatedinagraphwithcycles,thetraversalmightnotabortinasensibleamountoftime.

Optimizations

Thereareafewoptionsformakingatraversalrunfaster.

Thebestoptionistomaketheamountofvisitedverticesandfollowededgesassmallaspossible.Thiscanbeachievedbywritingcustomfilterandexpanderfunctions.Suchfunctionsshouldonlyincludeverticesofinterest,andonlyfollowedgesthatmightbeinteresting.

TraversaldepthcanalsobeboundedwiththeminDepthandmaxDepthoptions.

Anotherwaytospeeduptraversalsistowriteacustomvisitorfunction.Thedefaultvisitorfunction(trackingVisitor)willcopyeveryvisitedvertexintotheresult.Ifverticescontainlotsofdata,thismightbeexpensive.Itisthereforerecommendedtoonlycopysuchdataintotheresultthatisactuallyneeded.Thedefaultvisitorfunctionwillalsocopythefullpathtothevisiteddocumentintotheresult.Thisisevenmoreexpensiveandshouldbeavoidedifpossible.

Ifthegoalofatraversalistoonlycountthenumberofvisitedvertices,theprefabcountingVisitorwillbemuchmoreefficientthanthedefaultvisitor.

Forgraphsthatareknowntonotcontainanycycles,uniquenesschecksshouldbeturnedoff.Thiscanachievedviatheuniquenessconfigurationoptions.Notethatuniquenesschecksshouldnotbeturnedoffforgraphsthatareknowncontaincyclesorifthereisnoinformationaboutthegraph'sstructure.

Bydefault,atraversalwillonlyprocessalimitednumberofvertices.Thisisprotecttheuserfromunintentionallyrunanever-endingtraversalonagraphwithcyclicdata.HowmanyverticeswillbeprocessedatmostisdeterminedbythemaxIterationsconfigurationoption.IfatraversalhitsthecapspecifiedbymaxIterations,itwillabortandthrowatoomanyiterationsexception.Ifthiserrorisencountered,themaxIterationsvalueshouldbeincreasedifitismadesurethattheothertraversalconfigurationparametersaresaneandthetraversalwillabortnaturallyatsomepoint.

Finally,thebuildVerticesconfigurationoptioncanbesettofalsetoavoidlookingupandfullyconstructingvertexdata.Ifallthat'sneededfromverticesarethe_idor_keyattributes,thebuildverticesoptioncanbesettofalse.Ifvisitor,filterorexpandFilterfunctionsneedtoaccessothervertexattributes,theoptionshouldnotbechanged.

ConfigurationOverview

Thissectionsummarizestheconfigurationattributesforthetraversalobject.Theconfigurationcanconsistofthefollowingattributes:

visitor:visitorfunctionforvertices.Itwillbecalledforallnon-excludedvertices.Thegeneralvisitorfunctionsignatureisfunction(config,result,vertex,path).Ifthetraversalorderispreorder-expander,theconnectingedgesofthevisitedvertexwillbepassedasthefifthparameter,extendingthefunctionsignatureto:function(config,result,vertex,path,edges).

Visitorfunctionsarenotexpectedtoreturnvalues,buttheymaymodifytheresultvariableasneeded(e.g.bypushingvertexdataintotheresult).

expander:expanderfunctionthatisresponsibleforreturningedgesandverticesdirectlyconnectedtoavertex.Thefunctionsignatureisfunction(config,vertex,path).Theexpanderfunctionisrequiredtoreturnanarrayofconnectionobjects,consistingofanedgeandvertexattributeeach.Iftherearenoconnectingedges,theexpanderisexpectedtoreturnanemptyarray.filter:vertexfilterfunction.Thefunctionsignatureisfunction(config,vertex,path).Itmayreturnoneofthefollowingvalues:

undefined:vertexwillbeincludedintheresultandconnectededgeswillbetraversed"exclude":vertexwillnotbeincludedintheresultandconnectededgeswillbetraversed"prune":vertexwillbeincludedintheresultbutconnectededgeswillnotbetraversed["prune" ,"exclude" ]:vertexwillnotbeincludedintheresultandconnectededgeswillnotbereturned

expandFilter:filterfunctionappliedoneachedge/vertexcombinationdeterminedbytheexpander.Thefunctionsignatureisfunction(config,vertex,edge,path).Thefunctionshouldreturntrueiftheedge/vertexcombinationshouldbeprocessed,andfalseifitshouldbeignored.sort:afilterfunctiontodeterminetheorderinwhichconnectededgesareprocessed.Thefunctionsignatureisfunction(l,r).Thefunctionisrequiredtoreturnoneofthefollowingvalues:

-1iflshouldhaveasortvaluelessthanr

1iflshouldhaveahighersortvaluethanr0iflandrhavethesamesortvalue

strategy:determinesthevisitationstrategy.Possiblevaluesaredepthfirstandbreadthfirst.order:determinesthevisitationorder.Possiblevaluesarepreorder,postorder,andpreorder-expander.preorder-expanderisthesameaspreorder,exceptthatthesignatureofthevisitorfunctionwillchangeasdescribedabove.itemOrder:determinestheorderinwhichconnectionsreturnedbytheexpanderwillbeprocessed.Possiblevaluesareforwardandbackward.maxDepth:ifsettoavaluegreaterthan0,thiswilllimitthetraversaltothismaximumdepth.minDepth:ifsettoavaluegreaterthan0,allverticesfoundonalevelbelowtheminDepthlevelwillnotbeincludedintheresult.maxIterations:themaximumnumberofiterationsthatthetraversalisallowedtoperform.Itissensibletosetthisnumbersounboundedtraversalswillterminateatsomepoint.uniqueness:anobjectthatdefineshowrepeatedvisitationsofverticesshouldbehandled.Theuniquenessobjectcanhaveasub-attributevertices,andasub-attributeedges.Eachsub-attributecanhaveoneofthefollowingvalues:

"none":nouniquenessconstraints"path":elementisexcludedifitisalreadycontainedinthecurrentpath.Thissettingmaybesensibleforgraphsthatcontaincycles(e.g.A→B→C→A)."global":elementisexcludedifitwasalreadyfound/visitedatanypointduringthetraversal.

buildVertices:thisattributecontrolswhetherverticesencounteredduringthetraversalwillbelookedupinthedatabaseandwillbemadeavailabletovisitor,filter,andexpandFilterfunctions.Bydefault,verticeswillbelookedupandmadeavailable.However,therearesomespecialusecaseswhenfullyconstructingvertexobjectsisnotnecessaryandcanbeavoided.Forexample,ifatraversalismeanttoonlycountthenumberofvisitedverticesbutdonotreadanydatafromvertices,thisoptionmightbesettotrue.

ExampleDataThefollowingexamplesalluseavertexcollectionvandanedgecollectione.Thevertexcollectionvcontainscontinents,countries,andcapitals.Theedgecollectionecontainsconnectionsbetweencontinentsandcountries,andbetweencountriesandcapitals.

Tosetupthecollectionsandpopulatethemwithinitialdata,thefollowingscriptwasused:

db._create("v");

db._createEdgeCollection("e");

//vertices:rootnode

db.v.save({_key:"world",name:"World",type:"root"});

//vertices:continents

db.v.save({_key:"continent-africa",name:"Africa",type:"continent"});

db.v.save({_key:"continent-asia",name:"Asia",type:"continent"});

db.v.save({_key:"continent-australia",name:"Australia",type:"continent"});

db.v.save({_key:"continent-europe",name:"Europe",type:"continent"});

db.v.save({_key:"continent-north-america",name:"NorthAmerica",type:"continent"});

db.v.save({_key:"continent-south-america",name:"SouthAmerica",type:"continent"});

//vertices:countries

db.v.save({_key:"country-afghanistan",name:"Afghanistan",type:"country",code:"AFG"});

db.v.save({_key:"country-albania",name:"Albania",type:"country",code:"ALB"});

db.v.save({_key:"country-algeria",name:"Algeria",type:"country",code:"DZA"});

db.v.save({_key:"country-andorra",name:"Andorra",type:"country",code:"AND"});

db.v.save({_key:"country-angola",name:"Angola",type:"country",code:"AGO"});

db.v.save({_key:"country-antigua-and-barbuda",name:"AntiguaandBarbuda",type:"country",code:"ATG"});

db.v.save({_key:"country-argentina",name:"Argentina",type:"country",code:"ARG"});

db.v.save({_key:"country-australia",name:"Australia",type:"country",code:"AUS"});

db.v.save({_key:"country-austria",name:"Austria",type:"country",code:"AUT"});

db.v.save({_key:"country-bahamas",name:"Bahamas",type:"country",code:"BHS"});

db.v.save({_key:"country-bahrain",name:"Bahrain",type:"country",code:"BHR"});

db.v.save({_key:"country-bangladesh",name:"Bangladesh",type:"country",code:"BGD"});

db.v.save({_key:"country-barbados",name:"Barbados",type:"country",code:"BRB"});

db.v.save({_key:"country-belgium",name:"Belgium",type:"country",code:"BEL"});

db.v.save({_key:"country-bhutan",name:"Bhutan",type:"country",code:"BTN"});

db.v.save({_key:"country-bolivia",name:"Bolivia",type:"country",code:"BOL"});

db.v.save({_key:"country-bosnia-and-herzegovina",name:"BosniaandHerzegovina",type:"country",code:"BIH"});

db.v.save({_key:"country-botswana",name:"Botswana",type:"country",code:"BWA"});

db.v.save({_key:"country-brazil",name:"Brazil",type:"country",code:"BRA"});

db.v.save({_key:"country-brunei",name:"Brunei",type:"country",code:"BRN"});

db.v.save({_key:"country-bulgaria",name:"Bulgaria",type:"country",code:"BGR"});

db.v.save({_key:"country-burkina-faso",name:"BurkinaFaso",type:"country",code:"BFA"});

db.v.save({_key:"country-burundi",name:"Burundi",type:"country",code:"BDI"});

db.v.save({_key:"country-cambodia",name:"Cambodia",type:"country",code:"KHM"});

db.v.save({_key:"country-cameroon",name:"Cameroon",type:"country",code:"CMR"});

db.v.save({_key:"country-canada",name:"Canada",type:"country",code:"CAN"});

db.v.save({_key:"country-chad",name:"Chad",type:"country",code:"TCD"});

db.v.save({_key:"country-chile",name:"Chile",type:"country",code:"CHL"});

db.v.save({_key:"country-colombia",name:"Colombia",type:"country",code:"COL"});

db.v.save({_key:"country-cote-d-ivoire",name:"Coted'Ivoire",type:"country",code:"CIV"});

db.v.save({_key:"country-croatia",name:"Croatia",type:"country",code:"HRV"});

db.v.save({_key:"country-czech-republic",name:"CzechRepublic",type:"country",code:"CZE"});

db.v.save({_key:"country-denmark",name:"Denmark",type:"country",code:"DNK"});

db.v.save({_key:"country-ecuador",name:"Ecuador",type:"country",code:"ECU"});

db.v.save({_key:"country-egypt",name:"Egypt",type:"country",code:"EGY"});

db.v.save({_key:"country-eritrea",name:"Eritrea",type:"country",code:"ERI"});

db.v.save({_key:"country-finland",name:"Finland",type:"country",code:"FIN"});

db.v.save({_key:"country-france",name:"France",type:"country",code:"FRA"});

db.v.save({_key:"country-germany",name:"Germany",type:"country",code:"DEU"});

db.v.save({_key:"country-people-s-republic-of-china",name:"People'sRepublicofChina",type:"country",code:"CHN"});

//vertices:capitals

db.v.save({_key:"capital-algiers",name:"Algiers",type:"capital"});

db.v.save({_key:"capital-andorra-la-vella",name:"AndorralaVella",type:"capital"});

db.v.save({_key:"capital-asmara",name:"Asmara",type:"capital"});

db.v.save({_key:"capital-bandar-seri-begawan",name:"BandarSeriBegawan",type:"capital"});

db.v.save({_key:"capital-beijing",name:"Beijing",type:"capital"});

db.v.save({_key:"capital-berlin",name:"Berlin",type:"capital"});

db.v.save({_key:"capital-bogota",name:"Bogota",type:"capital"});

db.v.save({_key:"capital-brasilia",name:"Brasilia",type:"capital"});

ExampleData

db.v.save({_key:"capital-bridgetown",name:"Bridgetown",type:"capital"});

db.v.save({_key:"capital-brussels",name:"Brussels",type:"capital"});

db.v.save({_key:"capital-buenos-aires",name:"BuenosAires",type:"capital"});

db.v.save({_key:"capital-bujumbura",name:"Bujumbura",type:"capital"});

db.v.save({_key:"capital-cairo",name:"Cairo",type:"capital"});

db.v.save({_key:"capital-canberra",name:"Canberra",type:"capital"});

db.v.save({_key:"capital-copenhagen",name:"Copenhagen",type:"capital"});

db.v.save({_key:"capital-dhaka",name:"Dhaka",type:"capital"});

db.v.save({_key:"capital-gaborone",name:"Gaborone",type:"capital"});

db.v.save({_key:"capital-helsinki",name:"Helsinki",type:"capital"});

db.v.save({_key:"capital-kabul",name:"Kabul",type:"capital"});

db.v.save({_key:"capital-la-paz",name:"LaPaz",type:"capital"});

db.v.save({_key:"capital-luanda",name:"Luanda",type:"capital"});

db.v.save({_key:"capital-manama",name:"Manama",type:"capital"});

db.v.save({_key:"capital-nassau",name:"Nassau",type:"capital"});

db.v.save({_key:"capital-n-djamena",name:"N'Djamena",type:"capital"});

db.v.save({_key:"capital-ottawa",name:"Ottawa",type:"capital"});

db.v.save({_key:"capital-ouagadougou",name:"Ouagadougou",type:"capital"});

db.v.save({_key:"capital-paris",name:"Paris",type:"capital"});

db.v.save({_key:"capital-phnom-penh",name:"PhnomPenh",type:"capital"});

db.v.save({_key:"capital-prague",name:"Prague",type:"capital"});

db.v.save({_key:"capital-quito",name:"Quito",type:"capital"});

db.v.save({_key:"capital-saint-john-s",name:"SaintJohn's",type:"capital"});

db.v.save({_key:"capital-santiago",name:"Santiago",type:"capital"});

db.v.save({_key:"capital-sarajevo",name:"Sarajevo",type:"capital"});

db.v.save({_key:"capital-sofia",name:"Sofia",type:"capital"});

db.v.save({_key:"capital-thimphu",name:"Thimphu",type:"capital"});

db.v.save({_key:"capital-tirana",name:"Tirana",type:"capital"});

db.v.save({_key:"capital-vienna",name:"Vienna",type:"capital"});

db.v.save({_key:"capital-yamoussoukro",name:"Yamoussoukro",type:"capital"});

db.v.save({_key:"capital-yaounde",name:"Yaounde",type:"capital"});

db.v.save({_key:"capital-zagreb",name:"Zagreb",type:"capital"});

//edges:continent->world

db.e.save("v/continent-africa","v/world",{type:"is-in"});

db.e.save("v/continent-asia","v/world",{type:"is-in"});

db.e.save("v/continent-australia","v/world",{type:"is-in"});

db.e.save("v/continent-europe","v/world",{type:"is-in"});

db.e.save("v/continent-north-america","v/world",{type:"is-in"});

db.e.save("v/continent-south-america","v/world",{type:"is-in"});

//edges:country->continent

db.e.save("v/country-afghanistan","v/continent-asia",{type:"is-in"});

db.e.save("v/country-albania","v/continent-europe",{type:"is-in"});

db.e.save("v/country-algeria","v/continent-africa",{type:"is-in"});

db.e.save("v/country-andorra","v/continent-europe",{type:"is-in"});

db.e.save("v/country-angola","v/continent-africa",{type:"is-in"});

db.e.save("v/country-antigua-and-barbuda","v/continent-north-america",{type:"is-in"});

db.e.save("v/country-argentina","v/continent-south-america",{type:"is-in"});

db.e.save("v/country-australia","v/continent-australia",{type:"is-in"});

db.e.save("v/country-austria","v/continent-europe",{type:"is-in"});

db.e.save("v/country-bahamas","v/continent-north-america",{type:"is-in"});

db.e.save("v/country-bahrain","v/continent-asia",{type:"is-in"});

db.e.save("v/country-bangladesh","v/continent-asia",{type:"is-in"});

db.e.save("v/country-barbados","v/continent-north-america",{type:"is-in"});

db.e.save("v/country-belgium","v/continent-europe",{type:"is-in"});

db.e.save("v/country-bhutan","v/continent-asia",{type:"is-in"});

db.e.save("v/country-bolivia","v/continent-south-america",{type:"is-in"});

db.e.save("v/country-bosnia-and-herzegovina","v/continent-europe",{type:"is-in"});

db.e.save("v/country-botswana","v/continent-africa",{type:"is-in"});

db.e.save("v/country-brazil","v/continent-south-america",{type:"is-in"});

db.e.save("v/country-brunei","v/continent-asia",{type:"is-in"});

db.e.save("v/country-bulgaria","v/continent-europe",{type:"is-in"});

db.e.save("v/country-burkina-faso","v/continent-africa",{type:"is-in"});

db.e.save("v/country-burundi","v/continent-africa",{type:"is-in"});

db.e.save("v/country-cambodia","v/continent-asia",{type:"is-in"});

db.e.save("v/country-cameroon","v/continent-africa",{type:"is-in"});

db.e.save("v/country-canada","v/continent-north-america",{type:"is-in"});

db.e.save("v/country-chad","v/continent-africa",{type:"is-in"});

db.e.save("v/country-chile","v/continent-south-america",{type:"is-in"});

db.e.save("v/country-colombia","v/continent-south-america",{type:"is-in"});

db.e.save("v/country-cote-d-ivoire","v/continent-africa",{type:"is-in"});

db.e.save("v/country-croatia","v/continent-europe",{type:"is-in"});

db.e.save("v/country-czech-republic","v/continent-europe",{type:"is-in"});

db.e.save("v/country-denmark","v/continent-europe",{type:"is-in"});

db.e.save("v/country-ecuador","v/continent-south-america",{type:"is-in"});

ExampleData

db.e.save("v/country-egypt","v/continent-africa",{type:"is-in"});

db.e.save("v/country-eritrea","v/continent-africa",{type:"is-in"});

db.e.save("v/country-finland","v/continent-europe",{type:"is-in"});

db.e.save("v/country-france","v/continent-europe",{type:"is-in"});

db.e.save("v/country-germany","v/continent-europe",{type:"is-in"});

db.e.save("v/country-people-s-republic-of-china","v/continent-asia",{type:"is-in"});

//edges:capital->country

db.e.save("v/capital-algiers","v/country-algeria",{type:"is-in"});

db.e.save("v/capital-andorra-la-vella","v/country-andorra",{type:"is-in"});

db.e.save("v/capital-asmara","v/country-eritrea",{type:"is-in"});

db.e.save("v/capital-bandar-seri-begawan","v/country-brunei",{type:"is-in"});

db.e.save("v/capital-beijing","v/country-people-s-republic-of-china",{type:"is-in"});

db.e.save("v/capital-berlin","v/country-germany",{type:"is-in"});

db.e.save("v/capital-bogota","v/country-colombia",{type:"is-in"});

db.e.save("v/capital-brasilia","v/country-brazil",{type:"is-in"});

db.e.save("v/capital-bridgetown","v/country-barbados",{type:"is-in"});

db.e.save("v/capital-brussels","v/country-belgium",{type:"is-in"});

db.e.save("v/capital-buenos-aires","v/country-argentina",{type:"is-in"});

db.e.save("v/capital-bujumbura","v/country-burundi",{type:"is-in"});

db.e.save("v/capital-cairo","v/country-egypt",{type:"is-in"});

db.e.save("v/capital-canberra","v/country-australia",{type:"is-in"});

db.e.save("v/capital-copenhagen","v/country-denmark",{type:"is-in"});

db.e.save("v/capital-dhaka","v/country-bangladesh",{type:"is-in"});

db.e.save("v/capital-gaborone","v/country-botswana",{type:"is-in"});

db.e.save("v/capital-helsinki","v/country-finland",{type:"is-in"});

db.e.save("v/capital-kabul","v/country-afghanistan",{type:"is-in"});

db.e.save("v/capital-la-paz","v/country-bolivia",{type:"is-in"});

db.e.save("v/capital-luanda","v/country-angola",{type:"is-in"});

db.e.save("v/capital-manama","v/country-bahrain",{type:"is-in"});

db.e.save("v/capital-nassau","v/country-bahamas",{type:"is-in"});

db.e.save("v/capital-n-djamena","v/country-chad",{type:"is-in"});

db.e.save("v/capital-ottawa","v/country-canada",{type:"is-in"});

db.e.save("v/capital-ouagadougou","v/country-burkina-faso",{type:"is-in"});

db.e.save("v/capital-paris","v/country-france",{type:"is-in"});

db.e.save("v/capital-phnom-penh","v/country-cambodia",{type:"is-in"});

db.e.save("v/capital-prague","v/country-czech-republic",{type:"is-in"});

db.e.save("v/capital-quito","v/country-ecuador",{type:"is-in"});

db.e.save("v/capital-saint-john-s","v/country-antigua-and-barbuda",{type:"is-in"});

db.e.save("v/capital-santiago","v/country-chile",{type:"is-in"});

db.e.save("v/capital-sarajevo","v/country-bosnia-and-herzegovina",{type:"is-in"});

db.e.save("v/capital-sofia","v/country-bulgaria",{type:"is-in"});

db.e.save("v/capital-thimphu","v/country-bhutan",{type:"is-in"});

db.e.save("v/capital-tirana","v/country-albania",{type:"is-in"});

db.e.save("v/capital-vienna","v/country-austria",{type:"is-in"});

db.e.save("v/capital-yamoussoukro","v/country-cote-d-ivoire",{type:"is-in"});

db.e.save("v/capital-yaounde","v/country-cameroon",{type:"is-in"});

db.e.save("v/capital-zagreb","v/country-croatia",{type:"is-in"});

ExampleData

Edges,Identifiers,HandlesThisisanintroductiontoArangoDB'sinterfaceforedges.Edgesmaybeusedingraphs.HereweworkwithedgesfromtheJavaScriptshellarangosh.ForotherlanguagesseethecorrespondinglanguageAPI.

Agraphdatamodelalwaysconsistsofatleasttwocollections:therelationsbetweenthenodesinthegraphsarestoredinan"edgescollection",thenodesinthegrapharestoredindocumentsinregularcollections.

EdgesinArangoDBarespecialdocuments.Inadditiontothesystemattributes_key,_idand_rev,theyhavetheattributes_fromand_to,whichcontaindocumenthandles,namelythestart-pointandtheend-pointoftheedge.

Example:

the"edge"collectionstorestheinformationthatacompany'sreceptionissub-unittotheservicesunitandtheservicesunitissub-unittotheCEO.Youwouldexpressthisrelationshipwiththe_fromand_toattributesthe"normal"collectionstoresallthepropertiesaboutthereception,e.g.that20peopleareworkingthereandtheroomnumberetc_fromisthedocumenthandleofthelinkedvertex(incomingrelation)_toisthedocumenthandleofthelinkedvertex(outgoingrelation)

Edgecollectionsarespecialcollectionsthatstoreedgedocuments.Edgedocumentsareconnectiondocumentsthatreferenceotherdocuments.Thetypeofacollectionmustbespecifiedwhenacollectioniscreatedandcannotbechangedafterwards.

Tochangeedgeendpointsyouwouldneedtoremoveolddocument/edgeandinsertnewone.Otherfieldscanbeupdatedasindefaultcollection.

WorkingwithEdges

Edgesarenormaldocumentsthatalwayscontaina_fromanda_toattribute.

WorkingwithEdges

DistributedIterativeGraphProcessing(Pregel)Distributedgraphprocessingenablesyoutodoonlineanalyticalprocessingdirectlyongraphsstoredintoarangodb.Thisisintendedtohelpyougainanalyticalinsightsonyourdata,withouthavingtouseexternalprocessingsytems.ExamplesofalgorithmstoexecutearePageRank,VertexCentrality,VertexCloseness,ConnectedComponents,CommunityDetection.Thissystemisnotusefulfortypicalonlinequeries,whereyoujustdoworkonasmallsetofvertices.ThesekindoftasksarebettersuitedforAQL.

TheprocessingsysteminsideArangoDBisbasedon:Pregel:ASystemforLarge-ScaleGraphProcessing–Malewiczetal.(Google)2010Thisconceptenablesustoperformdistributedgraphprocessing,withouttheneedfordistributedgloballocking.

PrerequisitesIfyouarerunningasingleArangoDBinstanceinsingle-servermode,therearenorequirementsregardingthemodelingofyourdata.Allyouneedisatleastonevertexcollectionandoneedgecollection.Notethattheperformancemaybebetter,ifthenumberofyourshards/collectionsmatchesthenumberofCPUcores.

WhenyouuseArangoDBCommunityeditioninclustermode,youmightneedtomodelyourcollectionsinacertainwaytoensurecorrectresults.Formoreinformationseethenextsection.

RequirementsforCollectionsinaCluster(NonSmartGraph)Toenableiterativegraphprocessingforyourdata,youwillneedtoensurethatyourvertexandedgecollectionsareshardedinaspecificway.

ThepregelcomputingmodelrequiresalledgestobepresentontheDBServerwherethevertexdocumentidentifiedbythe_fromvalueislocated.Thismeansthevertexcollectionsneedtobeshardedby'_key'andtheedgecollectionwillneedtobeshardedafteranattributewhichalwayscontainsthe'_key'ofthevertex.

Ourimplementationcurrentlyrequireseveryedgecollectiontobeshardedaftera"vertex"attributes,additionallyyouwillneedtospecifythekeydistributeShardsLikeandanequalnumberofshardsoneverycollection.OnlyiftheserequirementsaremetcanArangoDBplacetheedgesandverticescorrectly.

Forexampleyoumightcreateyourcollectionslikethis:

//Createmainvertexcollection:

db._create("vertices",{

shardKeys:['_key'],

numberOfShards:8

//Optionallycreatearbitraryadditionalvertexcollections

db._create("additonal",{

distributeShardsLike:"vertices",

numberOfShards:8

//Create(oneormore)edge-collections:

db._createEdgeCollection("edges",{

shardKeys:['vertex'],

distributeShardsLike:"vertices",

numberOfShards:8

Youwillneedtoensurethatedgedocumentscontainthepropervaluesintheirshardingattribute.Foravertexdocumentwiththefollowingcontent{_key:"A",value:0}thecorrespondingedgedocumentswouldhavelooklikethis:

{_from:"vertices/A",_to:"vertices/B",vertex:"A"}

{_from:"vertices/A",_to:"vertices/C",vertex:"A"}

{_from:"vertices/A",_to:"vertices/D",vertex:"A"}

Pregel

ThiswillensurethatoutgoingedgedocumentswillbeplacedonthesameDBServerasthevertex.Withoutthecorrectplacementoftheedges,thepregelgraphprocessingsystemwillnotworkcorrectly,becauseedgeswillnotloadcorrectly.

ArangoshAPI

StartinganAlgorithmExecution

ThepregelAPIisaccessiblethroughthe@arangodb/pregelpackage.Tostartanexecutionyouneedtospecifythealgorithmnameandthevertexandedgecollections.Alternativelyyoucanspecifyanamedgraph.Additionallyyoucanspecifycustomparameterswhichvaryforeachalgorithm.ThestartmethodwillalwaysauniqueIDwhichcanbeusedtointeractwiththealgorithmandlateron.

Thebelowversionofthestartmethodcanbeusedfornamedgraphs:

varpregel=require("@arangodb/pregel");

varparams={};

varexecution=pregel.start("<algorithm>","<yourgraph>",params);

Paramsneedstobeanobject,thevalidkeysarementionedbelowinthesectionAlgorithms

Alternativelyyoumightwanttospecifythevertexandedgecollectionsdirectly.Thecall-syntaxofthestart``methodchangesinthiscase.ThesecondargumentmustbeanobjectwiththekeysvertexCollectionsandedgeCollections`.

varparams={};

varexecution=pregel.start("<algorithm>",{vertexCollections:["vertices"],edgeCollections:["edges"]},{});

Thelastargumentisstilltheparameterobject.Seebelowforalistofalgorithmsandparameters.

StatusofanAlgorithmExecution

Thecodereturnedbythepregel.start(...)methodcanbeusedtotrackthestatusofyouralgorithm.

varexecution=pregel.start("sssp","demograph",{source:"vertices/V"});

varstatus=pregel.status(execution);

Theresultwilltellyouthecurrentstatusofthealgorithmexecution.Itwilltellyouthecurrentstateoftheexecution,thecurrentglobalsuperstep,theruntime,theglobalaggregatorvaluesaswellasthenumberofsendandreceivedmessages.

Validvaluesforthestatefieldinclude:

"running"algorithmisstillrunning"done":Theexecutionisdone,theresultmightnotbewrittenbackintothecollectionyet."canceled":Theexecutionwaspermanentlycanceled,eitherbytheuserorbyanerror."inerror":Theexeuctionisinanerrorstate.ThiscanbecausedbyprimaryDBServersbeingnotreachableorbeingnonresponsive.Theexecutionmightrecoverlater,orswitchto"canceled"ifitwasnotabletorecoversuccessfuly"recovering":Theexecutionisactivelyrecovering,willswitchbackto"running"iftherecoverywassuccessful

Theobjectreturnedbythestatusmethodmightforexamplelooksomethinglikethis:

"state":"running",

"gss":12,

"totalRuntime":123.23,

"aggregators":{

"converged":false,

"max":true,

"phase":2

"sendCount":3240364978,

"receivedCount":3240364975

Pregel

CancelinganExecution/Discardingresults

Tocancelanexecutionwhichisstillrunnning,anddiscardanyintermediareresultsyoucanusethecancelmethod.Thiswillimmediatlyfreeallmemorytakenupbytheexecution,andwillmakeyouloseallintermediarydata.

Youmightgetinconsistentresultsifyoucancelanexecutionwhileitisalreadyinit'sdonestate.Thedataiswrittenmulti-threadedintoallcollectionshardsatonce,thismeanstherearemultipletransactionssimultaniously.Atransactionmightalreadybecommitedwhenyoucanceltheexecutionjob,thereforeyoumightseetheresultinyourcollection.Thisdoesnotapplyifyouconfiguredtheexecutiontonotwritedataintothecollection.

//startasinglesourceshortestpathjob

varexecution=pregel.start("sssp","demograph",{source:"vertices/V"});

pregel.cancel(execution);

AQLintegrationArangoDBsupportsretrievingtemporarypregelresultsthroughtheArangoDBquerylanguage(AQL).Whenourgraphprocessingsubsystemfinishesexecutinganalgorithm,theresultcaneitherbewrittenbackintothedatabaseorkeptinmemory.InbothcasestheresultcanbequeriedviaAQL.Ifthedatawasnotwrittentothedatabasestoreitisonlyheldtemporarily,untiltheusercallsthecancelmethodForexampleausermightwanttoqueryonlynodeswiththemostrankfromtheresultsetofaPageRankexecution.

FORvINPREGEL_RESULT(<handle>)

FILTERv.value>=0.01

RETURNv._key

AvailableAlgorithmsThereareanumberofgeneralparameterswhichapplytoalmostallalgorithms:

store:Isperdefaulttrue,thepregelenginewillwriteresultsbacktothedatabase.ifthevalueisfalsethenyoucanquerytheresultsviaAQLk,seeAQLintegration.maxGSS:Maximumnumberofglobaliterationsforthisalgorithmparallelism:Numberofparellelthreadstouseperworker.Doesnotinfluencethenumberofthreadsusedtoload

orstoredatafromthedatabase(thisdependsonthenumberofshards).

async:Algorithmswichsupportasyncmode,willrunwithoutsynchronizedglobaliterations,mightleadtoperformanceincreasesifyouhaveloadimbalances.resultField:Mostalgorithmswillwritetheresultintothisfield

PageRank

PageRankisawellknownalgorithmtorankdocumentsinagraph.Thealgorithmwillrununtiltheexecutionconverges.Specifyacustomthresholdwiththeparameterthreshold,torunforafixednumberofiterationsusethemaxGSSparameter.

pregel.start("pagerank","graphname",{maxGSS:100,threshold:0.00000001})

Single-SourceShortestPath

Calculatesthedistanceofeachvertextoacertainshortestpath.Thealgorithmwillrununtilitconverges,theiterationsareboundbythediameter(thelongestshortestpath)ofyourgraph.

pregel.start("sssp","graphname",{source:"vertices/1337"})

ConnectedComponents

Pregel

Therearetwoalgorithmstofindconnectedcomponentsinagraph.Tofindweaklyconnectedcomponents(WCC)youcanusethealgorithmnamed"connectedcomponents",tofindstronglyconnectedcomponents(SCC)youcanusethealgorithmnamed"scc".BothalgorithmwillassignacomponentIDtoeachvertex.

Aweaklyconnectedcomponentsmeansthatthereexistapathfromeveryvertexpairinthatcomponent.WCCisaverysimpleandfastalgorithm,whichwillonlyworkcorrectlyonundirectedgraphs.Yourresultsondirectedgraphsmayvary,dependingonhowconnectedyourcomponentsare.

InthecaseofSCCacomponentmeanseveryvertexisreachablefromanyothervertexinthesamecomponent.ThealgorithmismorecomplexthantheWCCalgorithmandrequiresmoreRAM,becauseeachvertexneedstostoremuchmorestate.ConsiderusingWCCifyouthinkyourdatamaybesuitableforit.

//weaklyconnectedcomponents

pregel.start("connectedcomponents","graphname")

//stronglyconnectedcomponents

pregel.start("scc","graphname")

Hyperlink-InducedTopicSearch(HITS)

HITSisalinkanalysisalgorithmthatratesWebpages,developedbyJonKleinberg(Thealgorithmisalsoknownashubsandauthorities).

TheideabehindHubsandAuthoritiescomesfromthetypicalstructureoftheweb:Certainwebsitesknownashubs,serveaslargedirectoriesthatarenotactuallyauthoritativeontheinformationthattheyhold.Thesehubsareusedascompilationsofabroadcatalogofinformationthatleadsusersdirecttootherauthoritativewebpages.Thealgorithmassignseachvertextwoscores:Theauthority-scoreandthehub-score.Theauthorityscorerateshowmanygoodhubspointtoaparticularvertex(orwebpage),thehubscorerateshowgood(authoritative)theverticespointedtoare.Formoreseehttps://en.wikipedia.org/wiki/HITS_algorithm

Ourversionofthealgorithmconvergesafteracertainamountoftime.Theparameterthresholdcanbeusedtosetalimitfortheconvergence(measuredasmaximumabsolutedifferenceofthehubandauthorityscoresbetweenthecurrentandlastiteration)Whenyouspecifytheresultfieldname,thehubscorewillbestoredin"_hub"andtheauthorityscorein"_auth".Thealgorithmcanbeexecutedlikethis:

varhandle=pregel.start("hits","yourgraph",{threshold:0.00001,resultField:"score"});

VertexCentrality

Centralitymeasureshelpidentifythemostimportantverticesinagraph.Theycanbeusedinawiderangeofapplications:Forexampletheycanbeusedtoidentifyinfluencersinsocialnetworks,ormiddle-meninterroristnetworks.Therearevariousdefinitionsforcentrality,thesimplestonebeingthevertexdegree.Thesedefinitionswerenotdesignedwithscalabilityinmind.Itisprobablyimpossibletodiscoveranefficientalgorithmwhichcomputestheminadistributedway.Fortunatelytherearescalablesubstitutionsavailable,whichshouldbeequallyusableformostusecases.

Pregel

EffectiveCloseness

Acommondefinitionsofcentralityistheclosenesscentrality(orcloseness).Theclosenessofavertexinagraphistheinverseaveragelengthoftheshortestpathbetweenthevertexandallothervertices.Forverticesx,yandshortestdistanced(y,x)itisdefinedas

EffectiveClosenessapproximatestheclosenessmeasure.Thealgorithmworksbyiterativelyestimatingthenumberofshortestpathspassingthrougheachvertex.Thescorewillapproximatesthetherealclosenessscore,sinceitisnotpossibletoactuallycountallshortestpathsduetothehorrendousO(n^2d)memoryrequirements.ThealgorithmisfromthepaperCentralitiesinLargeNetworks:AlgorithmsandObservations(UKanget.al.2011)*

ArangoDBsimplementationapproximatesthenumberofshortestpathineachiterationbyusingaHyperLogLogcounterwith64buckets.Thisshouldworkwellonlargegraphsandonsmalleronesaswell.ThememoryrequirementsshouldbeO(n*d)wherenisthenumberofverticesanddthediameterofyourgraph.Eachvertexwillstoreacounterforeachiterationofthealgorithm.Thealgorithmcanbeusedlikethis

constpregel=require("@arangodb/pregel");

consthandle=pregel.start("effectivecloseness","yourgraph",{resultField:"closeness"});

LineRank

Anothercommonmeasureisthebetweenness*centrality:Itmeasuresthenumberoftimesavertexispartofshortestpathsbetweenanypairsofvertices.Foravertexvbetweennessisdefinedas

Wheretheσrepresentsthenumberofshortestpathsbetweenxandy,andσ(v)representsthenumberofpathsalsopassingthroughavertexv.Byintuitionavertexwithhigherbetweenesscentralitywillhavemoreinformationpassingthroughit.

LineRankapproximatestherandomwalkbetweennessofeveryvertexinagraph.Thisistheprobabilitythatsomeonestartingonanarbitaryvertex,willvisitthisnodewhenherandomlychoosesedgestovisit.Thealgoruthmessentiallybuildsalinegraphoutofyourgraph(switchestheverticesandedges),andthencomputesascoresimilartoPageRank.Thiscanbeconsideredascalableequivalenttovertexbetweeness,whichcanbeexecuteddistributedlyinArangoDB.ThealgorithmisfromthepaperCentralitiesinLargeNetworks:AlgorithmsandObservations(UKanget.al.2011)

Pregel

consthandle=pregel.start("linerank","yourgraph",{"resultField":"rank"});

CommunityDetection

Graphsbasedonrealworldnetworksoftenhaveacommunitystructure.Thismeansitispossibletofindgroupsofverticessuchthateacheachvertexgroupisinternallymoredenselyconnectedthanoutsidethegroup.Thishasmanyapplicationswhenyouwanttoanalyzeyournetworks,forexampleSocialnetworksincludecommunitygroups(theoriginoftheterm,infact)basedoncommonlocation,interests,occupation,etc.

LabelPropagation

LabelPropagationcanbeusedtoimplementcommunitydetectiononlargegraphs.Theideaisthateachvertexshouldbeinthecommunitythatmostofhisneighboursarein.WeiterativelydeteminethisbyfirstassigningrandomCommunityID's.Theneachitertation,avertexwillsendit'scurrentcommunityIDtoallhisneighborvertices.TheneachvertexadoptsthecommunityIDhereceivedmostfrequentlyduringtheiteration.

Thealgorithmrunsuntilitconverges,whichlikelyneverreallyhappensonlargegraphs.Thereforeyouneedtospecifyamaximumiterationboundwhichsuitsyou.Thedefaultboundis500iterations,whichislikelytoolargeforyourapplication.Shouldworkbestonundirectedgraphs,resultsondirectedgraphsmightvarydependingonthedensityofyourgraph.

consthandle=pregel.start("labelpropagation","yourgraph",{maxGSS:100,resultField:"community"});

Speaker-ListenerLabelPropagation

TheSpeaker-listenerLabelPropagation(SLPA)canbeusedtoimplementcommunitydetection.Itworkssimilartothelabelpropagationalgorithm,butnoweverynodeadditionallyaccumulatesamemoryofobservedlabels(insteadofforgettingallbutonelabel).

Beforethealgorithmrun,everyvertexisinitializedwithanuniqueID(theinitialcommunitylabel).Duringtherunthreestepsareexecutedforeachvertex:

1. Currentvertexisthelistenerallotherverticesarespeakers2. Eachspeakersendsoutalabelfrommemory,wesendoutarandomlabelwithaprobabilityproportionaltothenumberoftimesthe

vertexobservedthelabel3. Thelistenerremembersoneofthelabels,wealwayschoosethemostfrequentlyobservedlabel

consthandle=pregel.start("slpa","yourgraph",{maxGSS:100,resultField:"community"});

YoucanalsoexecuteSLPAwiththemaxCommunitiesparametertolimitthenumberofouputcommunities.Internallythealgorithmwillstillkeepthememoryofalllabels,buttheoutputisreducedtojusthenmostfrequentlyobservedlabels.

consthandle=pregel.start("slpa","yourgraph",{maxGSS:100,resultField:"community",maxCommunities:1});

//checkthestatusperiodicallyforcompletion

pregel.status(handle);

Pregel

ViewsavailableinArangoDBArangoSearch

BringingthepowerofIResearchtoArangoDB

WhatisArangoSearch

ArangoSearchisanativelyintegratedAQLextensionmakinguseoftheIResearchlibrary.

Arangosearchallowsoneto:

joindocumentslocatedindifferentcollectionstooneresultlistfilterdocumentsbasedonAQLbooleanexpressionsandfunctionssorttheresultsetbasedonhowcloselyeachdocumentmatchedthefilter

Aconceptofvalue'analysis'thatismeanttobreakupagivenvalueintoasetofsub-valuesinternallytiedtogetherbymetadatawhichinfluencesboththefilterandsortstagestoprovidethemostappropriatematchforthespecifiedconditions,similartoqueriestowebsearchengines.

Inplaintermsthismeansausercanforexample:

requestdocumentswherethe'body'attributebestmatches'aquickbrownfox'requestdocumentswherethe'dna'attributebestmatchesaDNAsubsequencerequestdocumentswherethe'name'attributebestmatchesgenderetc...(viacustomanalyzersdescribedinthenextsection)

TheIResearchLibrary

IResearchsacross-platformopensourceindexingandsearchingenginewritteninC++,optimizedforspeedandmemoryfootprint,withsourceavailablefrom:https://github.com/iresearch-toolkit/iresearch

IResearchisaframeworkforindexing,filteringandsortingofdata.Theindexingstagecantreateachdataitemasanatomorusecustom'analyzers'tobreakthedataitemintosub-atomicpiecestiedtogetherwithinternallytrackedmetadata.

TheIResearchframeworkingeneralcanbefurtherextendedatruntimewithcustomimplementationsofanalyzers(usedduringtheindexingandfilteringstages)andscorers(usedduringthesortingstage)allowingfullcontroloverthebehaviouroftheengine.

ArangoSearchScorers:

ArangoSearchaccessesscorersdirectlybytheirinternalnames.Thename(inupper-case)ofthescoreristhefunctionnametobeusedinthe'SORT'section.Functionarguments,(excludingthefirstargument),areserializedasastringrepresentationofaJSONarrayandpasseddirectlytothecorrespondingscorer.ThefirstargumenttoanyscorerfunctionisthereferencetothecurrentdocumentemittedbytheFORstatement,i.e.itwouldbe'doc'forthisstatement:

FORdocINVIEWsomeView

IResearchprovidesa'bm25'scorerimplementingtheBM25algorithm.Thisscoreroptionallytakes'k'and'b'positionalparameters.

Theuserisabletorunquerieswiththesaidscorer,e.g.

SORTBM25(doc,1.2,0.75)

ThefunctionargumentswillthenbeserializedintoaJSONrepresentation:

[1.2,0.75]

andpassedtothescorerimplementation.

SimilarlyanadministratormaychoosetodeployacustomDNAanalyzer'DnaRank'.

Theuseristhenimmediatelyabletorunquerieswiththesaidscorer,e.g.

ArangoSearch

SORTDNARANK(doc,123,456,"abc",{"def","ghi"})

ThefunctionargumentswillthenbeserializedintoaJSONrepresentation:

[123,456,"abc",{"def","ghi"}]

andpassedtothescorerimplementation.

Runtime-pluggingfunctionalityforscoresisnotavaiableinArangoDBatthispointintime,soArangoDBcomeswithafewdefault-initializedscores:

attribute-nameorderresultsbasedonthevalueofattribute-name

BM25orderresultsbasedontheBM25algorithm

TFIDForderresultsbasedontheTFIDFalgorithm

ArangoSearchismuchmorethanafulltextsearch

Butfulltextsearchingisasubsetofitsavailablefunctionality,supportedviathe'text'analyzerand'tfidf'/'bm25'scorers,withoutimpacttoperformancewhenspecifyingdocumentsfromdifferentcollectionsorfilteringonmultipledocumentattributes.

Viewdatasource

TheIResearchfunctionalityisexposedtoArangoDBviathetheArangoSearchviewAPIbecausetheArangoSearchviewismerelyanidentitytransformationappliedontodocumentsstoredinlinkedcollectionsofthesameArangoDBdatabase.InplaintermsanArangoSearchviewonlyallowsfilteringandsortingofdocumentslocatedincollectionsofthesamedatabase.Thematchingdocumentsthemselvesarereturnedas-isfromtheircorrespondingcollections.

LinkstoArangoDBcollections

AconceptofanArangoDBcollection'link'isintroducedtoallowspecifyingwhichArangoDBcollectionsagivenArangoSearchViewshouldqueryfordocumentsandhowthesedocumentsshouldbequeried.

AnArangoSearchLinkisauni-directionalconnectionfromanArangoDBcollectiontoanArangoSearchviewdescribinghowdatacomingfromthesaidcollectionshouldbemadeavailableinthegivenview.EachArangoSearchLinkinanArangoSearchviewisuniquelyidentifiedbythenameoftheArangoDBcollectionitlinksto.AnArangoSearchviewmayhavezeroormorelinks,eachtoadistinctArangoDBcollection.SimilarlyanArangoDBcollectionmaybereferencedvialinksbyzeroormoredistinctArangoSearchviews.InplaintermsanygivenArangoSearchviewmaybelinkedtoanygivenArangoDBcollectionofthesamedatabasewithzerooratmostonelink.However,anyArangoSearchviewmaybelinkedtomultipledistinctArangoDBcollectionsandsimilarlyanyArangoDBcollectionmaybereferencedbymultipleArangoSearchviews.

ToconfigureanArangoSearchviewforconsiderationofdocumentsfromagivenArangoDBcollectionalinkdefinitionmustbeaddedtothepropertiesofthesaidArangoSearchviewdefiningthelinkparametersasperthesectionViewdefinition/modification.

Analyzers:

TosimplifyquerysyntaxArangoSearchprovidesaconceptofnamedanalyzerswhicharemerelyaliasesfortype+configurationofIResearchanalyzers.ManagementofnamedanalyzersisexposedviabothREST,GUIandJavaScriptAPIs,e.g.

Viewdefinition/modification

AnArangoSearchviewisconfiguredviaanobjectcontainingasetofview-specificconfigurationdirectivesandamapoflink-specificconfigurationdirectives.

Duringviewcreationthefollowingdirectivesapply:

id:(optional)thedesiredviewidentifiername:(required)theviewnametype:\thevalue"arangosearch"anyofthedirectivesfromthesectionViewproperties

ArangoSearch

Duringviewmodificationthefollowingdirectivesapply:

links:(optional)amappingofcollection-name/collection-identifiertooneof:linkcreation-linkdefinitionasperthesectionLinkpropertieslinkremoval-JSONkeywordnull(i.e.nullifyalinkifpresent)anyofthedirectivesfromthesectionmodifiableviewproperties

Viewproperties(modifiable)

commit:(optional;default:usedefaultsforallvalues)configureArangoSearchViewcommitpolicyforsingle-iteminserts/removals,e.g.whenaddingremovingdocumentsfromalinkedArangoDBcollection

cleanupIntervalStep:(optional;default:10;todisableuse:0)waitatleastthismanycommitsbetweenremovingunusedfilesintheArangoSearchdatadirectoryforthecasewheretheconsolidationpoliciesmergesegmentsoften(i.e.alotofcommit+consolidate),alowervaluewillcausealotofdiskspacetobewastedforthecasewheretheconsolidationpoliciesrarelymergesegments(i.e.fewinserts/deletes),ahighervaluewillimpactperformancewithoutanyaddedbenefits

commitIntervalMsec:(optional;default:60000;todisableuse:0)waitatleastcountmillisecondsbetweencommittingviewdatastorechangesandmakingdocumentsvisibletoqueriesforthecasewheretherearealotofinserts/updates,alowervaluewillcausetheviewnottoaccountforthem,(unlitcommit),andmemoryusagewouldcontinuetogrowforthecasewherethereareafewinserts/updates,ahighervaluewillimpactperformanceandwastediskspaceforeachcommitcallwithoutanyaddedbenefits

commitTimeoutMsec:(optional;default:5000;todisableuse:0)trytocommitasmuchaspossiblebeforecountmillisecondsforthecasewheretherearealotofinserts/updates,alowervaluewillcauseadelayintheviewaccountingforthem,dueskippingofsomecommitsforthecasewheretherearealotofinserts/updates,ahighervaluewillcausehighermemoryconsumptionbetweencommitsduetoaccumulationofdocumentmodificationswhileacommitisinprogress

consolidate:(optional;default:none)aper-policymappingofthresholdsintherange[0.0,1.0]todeterminedatastoresegmentmergecandidates,ifspecifiedthenonlythelistedpoliciesareused,keysareanyof:

bytes:(optional;fordefaultvaluesuseanemptyobject:{})

intervalStep:(optional,default:10;todisableuse:0)applyconsolidationpolicywitheveryNthcommit

threshold:(optional;default:0.85)consolidateIFF{threshold}>segment_bytes/(all_segment_bytes/#segments)

bytes_accum:(optional;fordefaultvaluesuse:{})

intervalStep:(optional;default:10;todisableuse:0)applyconsolidationpolicywitheveryNthcommit

threshold:(optional;default:0.85)consolidateIFF{threshold}>(segment_bytes+sum_of_merge_candidate_segment_bytes)/all_segment_bytes

count:(optional;fordefaultvaluesuse:{})

threshold:(optional;default:0.85)consolidateIFF{threshold}>segment_docs{valid}/(all_segment_docs{valid}/#segments)

fill:(optional)ifspecified,useemptyobjectfordefaultvalues,i.e.{}

threshold:(optional;default:0.85)consolidateIFF{threshold}>#segment_docs{valid}/(#segment_docs{valid}+#segment_docs{removed})

dataPath:(default:\/arangosearch-\)thefilesystempathwheretostorepersistedviewmetadata

locale:(optional;default:C)thedefaultlocaleusedfororderingprocessedattributenames

threadsMaxIdle:(optional;default:5)maximumidlenumberofthreadsforsingle-runtasksforthecasewheretherearealotofshort-livedasynchronoustasks,alowervaluewillcausealotofthreadcreation/deletioncallsforthecasewheretherearenoshort-livedasynchronoustasks,ahighervaluewillonlywastememory

ArangoSearch

threadsMaxTotal:(optional;default:5)maximumtotalnumberofthreads(>0)forsingle-runtasksforthecasewheretherearealotofparallelizabletasksandanabundanceofresources,alowervaluewouldlimitperformanceforthecasewheretherearelimitedresourcesCPU/memory,ahighervaluewillnegativelyimpactperformance

Viewproperties(unmodifiable)

collections:aninternallytrackedlistofcollectionidentifierswhichwereexplicitlyaddedtothecurrentviewbytheuserviaview'link'propertymodificationthelistmayhaveno-longervalididentifiersiftheuserdidnotexplicitlydropthelinkforthesaidcollectionidentifierfromthecurrentviewinvalidcollectionidentifiersareremovedduringviewpropertymodificationamongotherthingsusedforacquiringcollectionlocksintransactions(i.e.duringaviewquerynodocumentswillbereturnedforcollectionsnotinthislist)andgeneratingviewproperties'links'list

Linkproperties

analyzers:(optional;default:['identity'])alistofanalyzers,bynameasdefinedviatheAnalyzers,thatshouldbeappliedtovaluesofprocesseddocumentattributes

fields:(optional;default:{})anobject{attribute-name:[Linkproperties]}offieldsthatshouldbeprocessedateachlevelofthedocumenteachkeyspecifiesthedocumentattributetobeprocessed,thevalueofincludeAllFieldsisalsoconsultedwhenselectingfieldstobeprocessedeachvaluespecifiestheLinkpropertiesdirectivestobeusedwhenprocessingthespecifiedfield,aLinkpropertiesvalueof{}denotesinheritanceofall(exceptfields)directivesfromthecurrentlevel

includeAllFields:(optional;default:false)iftruethenprocessalldocumentattributes(ifnotexplicitlyspecifiedthenprocessthefieldswithdefaultLinkpropertiesdirectives,i.e.{}),otherwiseonlyconsiderattributesmentionedinfields

trackListPositions:(optional;default:false)iftruethenforarrayvaluestrackthevaluepositioninthearray,e.g.whenqueryingfortheinput:{attr:['valueX','valueY','valueZ']}theusermustspecify:doc.attr[1]=='valueY'otherwiseallvaluesinanarrayaretreatedasequalalternatives,e.g.whenqueryingfortheinput:{attr:['valueX','valueY','valueZ']}theusermustspecify:doc.attr=='valueY'

ArangoSearch

Analyzers:

TosimplifyquerysyntaxArangoSearchprovidesaconceptofnamedanalyzerswhicharemerelyaliasesfortype+configurationofIResearchanalyzers.ManagementofnamedanalyzersisexposedviabothREST,GUIandJavaScriptAPIs,e.g.

db._globalSettings("iresearch.analyzers")

AuserthenmerelyusestheseanalyzernamesinArangoSearchviewconfigurationsandAQLqueries,e.g.

ArangoSearchprovidesa'text'analyzertoanalyzehumanreadabletext.Arequiredconfigurationparameterforthistypeofanalyzeris'locale'usedtospecifythelanguageusedforanalysis.

TheArangoDBadministratormaythensetupanamedanalyzer'text_des':

"name":"text_des",

"type":"text",

"properties":{

"locale":"de-ch"

Theuseristhenimmediatelyabletorunquerieswiththesaidanalyzer,e.g.

FILTERdoc.descriptionINTOKENS('EinbraunerFuchsspringt','text_des')

SimilarlyanadministratormaychoosetodeployacustomDNAanalyzer'DnaSeq':

"name":"dna",

"type":"DnaSeq",

"properties":"use-human-config"

Theuseristhenimmediatelyabletorunquerieswiththesaidanalyzer,e.g.

FILTERdoc.dnaINTOKENS('ACGTCGTATGCACTGA','DnaSeq')

Toalimiteddegreetheconceptof'analysis'isevenavailableinnon-IResearchAQL,e.g.theTOKENS(...)functionwillutilizethepowerofIResearchtobreakupavalueintoanAQLarraythatcanbeusedanywhereintheAQLquery.

Inplaintermsthismeansausercanmatchadocumentattributewhenitsvaluematchesatleastonevalueformaset,(yesthisisindependentofdoc),e.g.tomatchdocswith'word==quick'OR'word==brown'OR'word==fox'

FORdocINsomeCollection

FILTERdoc.wordINTOKENS('aquickbrownfox','text_en')

RETRUNdoc

Runtime-pluggingfunctionalityforanalyzersisnotavaiableinArangoDBatthispointintime,soArangoDBcomeswithafewdefault-initializedanalyzers:

identitytreatthevalueasanatom

text_detokenizethevalueintocase-insensitivewordstemsaspertheGermanlocale,donotdiscardanyanystopwords

text_entokenizethevalueintocase-insensitivewordstemsaspertheEnglishlocale,donotdiscardanyanystopwords

text_estokenizethevalueintocase-insensitivewordstemsaspertheSpanishlocale,donotdiscardanyanystopwords

text_fitokenizethevalueintocase-insensitivewordstemsaspertheFinnishlocale,donotdiscardanyanystopwords

text_frtokenizethevalueintocase-insensitivewordstemsaspertheFrenchlocale,donotdiscardanyanystopwords

text_ittokenizethevalueintocase-insensitivewordstemsaspertheItalianlocale,donotdiscardanyanystopwords

text_nltokenizethevalueintocase-insensitivewordstemsaspertheDutchlocale,donotdiscardanyanystopwords

text_notokenizethevalueintocase-insensitivewordstemsaspertheNorwegianlocale,donotdiscardanyanystopwords

Analyzers

text_pttokenizethevalueintocase-insensitivewordstemsasperthePortugueselocale,donotdiscardanyanystopwords

text_rutokenizethevalueintocase-insensitivewordstemsaspertheRussianlocale,donotdiscardanyanystopwords

text_svtokenizethevalueintocase-insensitivewordstemsaspertheSwedishlocale,donotdiscardanyanystopwords

text_zhtokenizethevalueintowordstemsaspertheChineselocale

Analyzers

FoxxTraditionally,server-sideprojectshavebeendevelopedasstandaloneapplicationsthatguidethecommunicationbetweentheclient-sidefrontendandthedatabasebackend.Thishasledtoapplicationsthatwereeitherdevelopedassinglemonolithsorthatduplicateddataaccessanddomainlogicacrossallservicesthathadtoaccessthedatabase.Additionally,toolstoabstractawaytheunderlyingdatabasecallscouldincuralotofnetworkoverheadwhenusingremotedatabaseswithoutcarefuloptimization.

ArangoDBallowsapplicationdeveloperstowritetheirdataaccessanddomainlogicasmicroservicesrunningdirectlywithinthedatabasewithnativeaccesstoin-memorydata.TheFoxxmicroserviceframeworkmakesiteasytoextendArangoDB'sownRESTAPIwithcustomHTTPendpointsusingmodernJavaScriptrunningonthesameV8engineyouknowfromNode.jsandtheGoogleChromewebbrowser.

Unliketraditionalapproachestostoringlogicinthedatabase(likestoredprocedures),thesemicroservicescanbewrittenasregularstructuredJavaScriptapplicationsthatcanbeeasilydistributedandversioncontrolled.Dependingonyourproject'sneedsFoxxcanbeusedtobuildanythingfromoptimizedRESTendpointsperformingcomplexdataaccesstoentirestandaloneapplicationsrunningdirectlyinsidethedatabase.

FoxxMicroservices

FoxxataglanceEachFoxxserviceisdefinedbyaJSONmanifestspecifyingtheentrypoint,anyscriptsdefinedbytheservice,possibleconfigurationoptionsandFoxxdependencies,aswellasothermetadata.Withinaservice,theseoptionsareexposedastheservicecontext.

AttheheartoftheFoxxframeworkliestheFoxxRouterwhichisusedtodefineHTTPendpoints.AservicecanaccessthedatabaseeitherdirectlyfromitscontextusingprefixedcollectionsortheArangoDBdatabaseAPI.

WhileFoxxisprimarilydesignedtobeusedtoaccessthedatabaseitself,ArangoDBalsoprovidesanAPItomakeHTTPrequeststoexternalservices.

Scriptscanbeusedtoperformone-offtasks,whichcanalsobescheduledtobeperformedasynchronouslyusingthebuilt-injobqueue.

Finally,FoxxservicescanbeinstalledandmanagedovertheWeb-UIorthroughArangoDBsHTTPAPI.

HowdoesitworkFoxxservicesconsistofJavaScriptcoderunningintheV8JavaScriptruntimeembeddedinsideArangoDB.EachserviceismountedineachavailableV8context(thenumberofcontextscanbeadjustedintheArangoDBconfiguration).Incomingrequestsaredistributedaccrossthesecontextsautomatically.

Ifyou'recomingfromanotherJavaScriptenvironmentlikeNode.jsthisissimilartorunningmultipleNode.jsprocessesbehindaloadbalancer:youshouldnotrelyonserver-sidestate(otherthanthedatabaseitself)betweendifferentrequestsasthereisnowayofmakingsureconsecutiverequestswillbehandledinthesamecontext.

BecausetheJavaScriptcodeisrunninginsidethedatabaseanotherdifferenceisthatallFoxxandArangoDBAPIsarepurelysynchronousandshouldbeconsideredblocking.Thisisespeciallyimportantfortransactions,whichinArangoDBcanexecutearbitrarycodebutmayhavetolockentirecollections(effectivelypreventinganydatatobewritten)untilthecodehascompleted.

Forinformationonhowthisaffectsinteroperabilitywiththird-partyJavaScriptmoduleswrittenforotherJavaScriptenvironmentsseethechapterondependencies.

DevelopmentmodeDevelopmentmodeallowsyoutomakechangestodeployedservicesin-placedirectlyonthedatabaseserver'sfilesystemwithoutdownloadingandre-uploadingtheservicebundle.Additionallyerrormessageswillcontainstacktraces.

YoucantoggledevelopmentmodeonandoffintheservicesettingstabofthewebinterfaceorusingtheHTTPAPI.Onceactivatedtheservice'sfilesystempathwillbeshownintheinfotab.

Onceenabledtheservice'ssourcefilesandmanifestwillbere-evaluated,andthesetupscript(ifpresent)re-executed,everytimearouteoftheserviceisaccessed,effectivelyre-deployingtheserviceoneveryrequest.Asthenameindicatesthisisintendedtobeusedstrictlyduringdevelopmentandismostdefinitelyabadideaonproductionservers.Theadditionalinformationexposedduringdevelopmentmodemayincludefilesystempathsandpartsoftheservice'ssourcecode.

Alsonotethatifyouareservingstaticfilesaspartofyourservice,accessingthesefilesfromabrowsermayalsotriggerare-deploymentoftheservice.Finally,makingHTTPrequeststoaservicerunningindevelopmentmodefromwithintheservice(i.e.usingthe@arangodb/requestmoduletoaccesstheserviceitself)isprobablynotagoodideaeither.

Bewareofdeletingthedatabasetheserviceisdeployedon:itwillerasethesourcefilesoftheservicealongwiththecollections.Youshouldbackupthecodeyouworkedonindevelopmentbeforedoingthattoavoidlosingyourprogress.

FoxxstoreTheFoxxstoreprovidesaccesstoanumberofready-to-useofficialandcommunity-maintainedFoxxservicesyoucaninstallwithasingleclick,includingexampleservicesandwrappersforexternalSaaStoolsliketransactionale-mailservices,bugloggersoranalyticstrackers.

YoucanfindtheFoxxstoreinthewebinterfacebyusingtheAddServicebuttonintheservicelist.

Ataglance

Cluster-Foxx

WhenrunningArangoDBinaclustertheFoxxserviceswillrunoneachcoordinator.Installing,upgradinganduninstallingservicesonanycoordinatorwillautomaticallydistributetheservicetotheothercoordinators,makingdeploymentsaseasyasinsingle-servermode.However,thismeanstherearesomelimitations:

Youshouldavoidanykindoffilesystemstatebeyondthedeployedservicebundleitself.Don'twritedatatothefilesystemorencodeanyexpectationsofthefilesystemstateotherthanthefilesintheservicefolderthatwereinstalledaspartoftheservice(e.g.fileuploadsorcustomlogfiles).

Additionally,thedevelopmentmodewillleadtoaninconsistentstateoftheclusteruntilitisdisabled.WhileaserviceisrunningindevelopmentmodeyoucanmakechangestotheserviceonthefilesystemofanycoordinatorandseethemreflectedinrealtimejustlikewhenrunningArangoDBasasingleserver.Howeverthechangesmadeononecoordinatorwillnotbereflectedacrossothercoordinatorsuntilthedevelopmentmodeisdisabled.Whendisablingthedevelopmentmodeforaservice,thecoordinatorwillcreateanewbundleanddistributeitacrosstheservicelikeamanualupgradeoftheservice.

Forthesereasonsitisstronglyrecommendednottousedevelopmentmodeinaclusterwithmultiplecoordinatorsunlessyouaresurethatnorequestsorchangeswillbemadetoothercoordinatorswhileyouaremodifyingtheservice.Usingdevelopmentmodeinaproductionclusterisextremelyunsafeandhighlydiscouraged.

Ataglance

GettingStartedWe'regoingtostartwithanemptyfolder.Thiswillbetherootfolderofourservices.Youcannameitsomethingcleverbutforthecourseofthisguidewe'llassumeit'scalledthenameofyourservice:getting-started.

Firstweneedtocreateamanifest.Createanewfilecalledmanifest.jsonandaddthefollowingcontent:

"engines":{

"arangodb":"^3.0.0"

ThisjusttellsArangoDBtheserviceiscompatiblewithversions3.0.0andlater(allthewayuptobutnotincluding4.0.0),allowingolderversionsofArangoDBtounderstandthatthisservicelikelywon'tworkforthemandnewerversionswhatbehaviortoemulateshouldtheystillsupportit.

Thelittlehattotheleftoftheversionnumberisnotatypo,it'scalleda"caret"andindicatestheversionrange.Foxxusessemanticversioning(alsocalled"semver")formostofitsversionhandling.Youcanfindoutmoreabouthowsemverworksattheofficialsemverwebsite.

Nextwe'llneedtospecifyanentrypointtoourservice.ThisistheJavaScriptfilethatwillbeexecutedtodefineourservice'sHTTPendpoints.Wecandothisbyaddinga"main"fieldtoourmanifest:

"engines":{

"arangodb":"^3.0.0"

"main":"index.js"

That'sallweneedinourmanifestfornow,solet'snextcreatetheindex.jsfile:

'usestrict';

constcreateRouter=require('@arangodb/foxx/router');

constrouter=createRouter();

module.context.use(router);

Thefirstlinecausesourfiletobeinterpretedusingstrictmode.AllexamplesintheArangoDBdocumentationassumestrictmode,soyoumightwanttofamiliarizeyourselfwithitifyouhaven'tencountereditbefore.

Thesecondlineimportsthe@arangodb/foxx/routermodulewhichprovidesafunctionforcreatingnewFoxxrouters.We'reusingthisfunctiontocreateanewrouterobjectwhichwe'llbeusingforourservice.

Themodule.contextistheso-calledFoxxcontextorservicecontext.ThisvariableisavailableinallfilesthatarepartofyourFoxxserviceandprovidesaccesstoFoxxAPIsspecifictothecurrentservice,liketheusemethod,whichtellsFoxxtomounttherouterinthisservice(andtoexposeitsroutestoHTTP).

Nextlet'sdefinearoutethatprintsagenericgreeting:

//continued

router.get('/hello-world',function(req,res){

res.send('HelloWorld!');

.response(['text/plain'],'Agenericgreeting.')

.summary('Genericgreeting')

.description('Printsagenericgreeting.');

Therouterprovidesthemethodsget,post,etccorrespondingtoeachHTTPverbaswellasthecatch-allall.ThesemethodsindicatethatthegivenrouteshouldbeusedtohandleincomingrequestswiththegivenHTTPverb(oranymethodwhenusingall).

Gettingstarted

Thesemethodstakeanoptionalpath(ifomitted,itdefaultsto"/")aswellasarequesthandler,whichisafunctiontakingthereq(request)andres(response)objectstohandletheincomingrequestandgeneratetheoutgoingresponse.IfyouhaveusedtheexpressframeworkinNode.js,youmayalreadybefamiliarwithhowthisworks,otherwisecheckoutthechapteronroutes.

Theobjectreturnedbytherouter'smethodsprovidesadditionalmethodstoattachmetadataandvalidationtotheroute.We'reusingsummaryanddescriptiontodocumentwhattheroutedoes--thesearen'tstrictlynecessarybutgiveussomeniceauto-generateddocumentation.Theresponsemethodletsusadditionallydocumenttheresponsecontenttypeandwhattheresponsebodywillrepresent.

Tryitout

AtthispointyoucanuploadtheservicefolderasaziparchivefromthewebinterfaceusingtheServicestab.

ClickAddServicethenpicktheZipoptioninthedialog.Youwillneedtoprovideamountpath,whichistheURLprefixatwhichtheservicewillbemounted(e.g./getting-started).

Onceyouhavepickedtheziparchiveusingthefilepicker,theuploadshouldbeginimmediatelyandyourserviceshouldbeinstalled.OtherwisepresstheInstallbuttonandwaitforthedialogtodisappearandtheservicetoshowupintheservicelist.

Clickanywhereonthecardwithyourmountpathonthelabeltoopentheservice'sdetails.

IntheAPIdocumentationyoushouldseetheroutewedefinedearlier(/hello-world)withthewordGETnexttoitindicatingtheHTTPmethoditsupportsandthesummaryweprovidedontheright.Byclickingontheroute'spathyoucanopenthedocumentationfortheroute.

Notethatthedescriptionweprovidedappearsinthegenerateddocumentationaswellasthedescriptionweaddedtotheresponse(whichshouldcorrectlyindicatethecontenttypetext/plain,i.e.plaintext).

ClicktheTryitout!buttontosendarequesttotherouteandyoushouldseeanexamplerequestwiththeservice'sresponse:"HelloWorld!".

Congratulations!Youhavejustcreated,installedandusedyourfirstFoxxservice.

Parametervalidation

Let'saddanotherroutethatprovidesamorepersonalizedgreeting:

//continued

constjoi=require('joi');

router.get('/hello/:name',function(req,res){

res.send(`Hello${req.pathParams.name}`);

.pathParam('name',joi.string().required(),'Nametogreet.')

.response(['text/plain'],'Apersonalizedgreeting.')

.summary('Personalizedgreeting')

.description('Printsapersonalizedgreeting.');

ThefirstlineimportsthejoimodulefromnpmwhichcomesbundledwithArangoDB.JoiisavalidationlibrarythatisusedthroughoutFoxxtodefineschemasandparametertypes.

Note:Youcanbundleyourownmodulesfromnpmbyinstallingtheminyourservicefolderandmakingsurethenode_modulesfolderisincludedinyourziparchive.Formoreinformationseethesectiononmoduledependenciesinthechapterondependencies.

ThepathParammethodallowsustospecifyparametersweareexpectinginthepath.Thefirstargumentcorrespondstotheparameternameinthepath,thesecondargumentisajoischematheparameterisexpectedtomatchandthefinalargumentservestodescribetheparameterintheAPIdocumentation.

ThepathparametersareaccessiblefromthepathParamspropertyoftherequestobject.We'reusingatemplatestringtogeneratetheserver'sresponsecontainingtheparameter'svalue.

Gettingstarted

NotethatrouteswithpathparametersthatfailtovalidatefortherequestURLwillbeskippedasiftheywouldn'texist.Thisallowsyoutodefinemultipleroutesthatareonlydistinguishedbytheschemasoftheirpathparameters(e.g.aroutetakingonlynumericparametersandonetakinganystringasafallback).

Let'stakethisfurtherandcreatearoutethattakesaJSONrequestbody:

//continued

router.post('/sum',function(req,res){

constvalues=req.body.values;

res.send({

result:values.reduce(function(a,b){

returna+b;

.body(joi.object({

values:joi.array().items(joi.number().required()).required()

}).required(),'Valuestoaddtogether.')

.response(joi.object({

result:joi.number().required()

}).required(),'Sumoftheinputvalues.')

.summary('Addupnumbers')

.description('Calculatesthesumofanarrayofnumbervalues.');

Notethatweusedposttodefinethisrouteinsteadofget(whichdoesnotsupportrequestbodies).TryingtosendaGETrequesttothisroute'sURL(intheabsenceofagetrouteforthesamepath)willresultinFoxxrespondingwithanappropriateerrorresponse,indicatingthesupportedHTTPmethods.

AsthisroutenotonlyexpectsaJSONobjectasinputbutalsorespondswithaJSONobjectasoutputweneedtodefinetwoschemas.Wedon'tstrictlyneedaresponseschemabutithelpsdocumentingwhattherouteshouldbeexpectedtorespondwithandwillshowupintheAPIdocumentation.

Becausewe'repassingaschematotheresponsemethodwedon'tneedtoexplicitlytellFoxxwearesendingaJSONresponse.ThepresenceofaschemaintheabsenceofacontenttypealwaysimplieswewantJSON.Thoughwecouldjustadd["application/json"]asanadditionalargumentaftertheschemaifwewantedtomakethismoreexplicit.

Thebodymethodworksthesamewayastheresponsemethodexcepttheschemawillbeusedtovalidatetherequestbody.Iftherequestbodycan'tbeparsedasJSONordoesn'tmatchtheschema,Foxxwillrejecttherequestwithanappropriateerrorresponse.

Creatingcollections

TherealpowerofFoxxcomesfrominteractingwiththedatabaseitself.Inordertobeabletouseacollectionfromwithinourservice,weshouldfirstmakesurethatthecollectionactuallyexists.Therightplacetocreatecollectionsyourserviceisgoingtouseisinasetupscript,whichFoxxwillexecuteforyouwheninstallingorupdatingtheservice.

Firstcreateanewfoldercalled"scripts"intheservicefolder,whichwillbewhereourscriptsaregoingtolive.Forsimplicity'ssake,oursetupscriptwillliveinafilecalledsetup.jsinsidethatfolder:

//continued

'usestrict';

constdb=require('@arangodb').db;

constcollectionName='myFoxxCollection';

if(!db._collection(collectionName)){

db._createDocumentCollection(collectionName);

Thescriptusesthedbobjectfromthe@arangodbmodule,whichletsusinteractwiththedatabasetheFoxxservicewasinstalledinandthecollectionsinsidethatdatabase.Becausethescriptmaybeexecutedmultipletimes(i.e.wheneverweupdatetheserviceorwhentheserverisrestarted)weneedtomakesurewedon'taccidentallytrytocreatethesamecollectiontwice(whichwouldresultinanexception);wedothatbyfirstcheckingwhetheritalreadyexistsbeforecreatingit.

Gettingstarted

The_collectionmethodlooksupacollectionbynameandreturnsnullifnocollectionwiththatnamewasfound.The_createDocumentCollectionmethodcreatesanewdocumentcollectionbyname(_createEdgeCollectionalsoexistsandworksanalogouslyforedgecollections).

Note:Becausewehavehardcodedthecollectionname,multiplecopiesoftheserviceinstalledalongsideeachotherinthesamedatabasewillsharethesamecollection.Becausethismaynotalwaysbewhatyouwant,theFoxxcontextalsoprovidesthecollectionNamemethodwhichappliesamountpointspecificprefixtoanygivencollectionnametomakeituniquetotheservice.Italsoprovidesthecollectionmethod,whichbehavesalmostexactlylikedb._collectionexceptitalsoappliestheprefixbeforelookingthecollectionup.

Nextweneedtotellourserviceaboutthescriptbyaddingittothemanifestfile:

"engines":{

"arangodb":"^3.0.0"

"main":"index.js",

"scripts":{

"setup":"scripts/setup.js"

Theonlythingthathaschangedisthatweaddeda"scripts"fieldspecifyingthepathofthesetupscriptwejustwrote.

Gobacktothewebinterfaceandupdatetheservicewithournewcode,thenchecktheCollectionstab.Ifeverythingworkedright,youshouldseeanewcollectioncalled"myFoxxCollection".

AccessingcollectionsLet'sexpandourservicebyaddingafewmoreroutestoourindex.js:

//continued

consterrors=require('@arangodb').errors;

constfoxxColl=db._collection('myFoxxCollection');

constDOC_NOT_FOUND=errors.ERROR_ARANGO_DOCUMENT_NOT_FOUND.code;

router.post('/entries',function(req,res){

constdata=req.body;

constmeta=foxxColl.save(req.body);

res.send(Object.assign(data,meta));

.body(joi.object().required(),'Entrytostoreinthecollection.')

.response(joi.object().required(),'Entrystoredinthecollection.')

.summary('Storeanentry')

.description('Storesanentryinthe"myFoxxCollection"collection.');

router.get('/entries/:key',function(req,res){

constdata=foxxColl.document(req.pathParams.key);

res.send(data)

}catch(e){

if(!e.isArangoError||e.errorNum!==DOC_NOT_FOUND){

throwe;

res.throw(404,'Theentrydoesnotexist',e);

.pathParam('key',joi.string().required(),'Keyoftheentry.')

.response(joi.object().required(),'Entrystoredinthecollection.')

.summary('Retrieveanentry')

.description('Retrievesanentryfromthe"myFoxxCollection"collectionbykey.');

We'reusingthesaveanddocumentmethodsofthecollectionobjecttostoreandretrievedocumentsinthecollectionwecreatedinoursetupscript.Becausewedon'tcarewhatthedocumentslooklikeweallowanyattributesontherequestbodyandjustacceptanobject.

Gettingstarted

BecausethekeywillbeautomaticallygeneratedbyArangoDBwhenonewasn'tspecifiedintherequestbody,we'reusingObject.assigntoapplytheattributesofthemetadataobjectreturnedbythesavemethodtothedocumentbeforereturningitfromourfirstroute.

Thedocumentmethodreturnsadocumentinacollectionbyits_keyor_id.HoweverwhennomatchingdocumentexistsitthrowsanArangoErrorexception.BecausewewanttoprovideamoredescriptiveerrormessagethanArangoDBdoesoutofthebox,weneedtohandlethaterrorexplicitly.

AllArangoErrorexceptionshaveatruthyattributeisArangoErrorthathelpsyourecognizingtheseerrorswithouthavingtoworryaboutinstanceofchecks.TheyalsoprovideanerrorNumandanerrorMessage.Ifyouwanttocheckforspecificerrorsyoucanjustimporttheerrorsobjectfromthe@arangodbmoduleinsteadofhavingtomemorizenumericerrorcodes.

Insteadofdefiningourownresponselogicfortheerrorcasewejustuseres.throw,whichmakestheresponseobjectthrowanexceptionFoxxcanrecognizeandconverttotheappropriateserverresponse.WealsopassalongtheexceptionitselfsoFoxxcanprovidemorediagnosticinformationwhenwewantitto.

Wecouldextendthepostroutetosupportarraysofobjectsaswell,eachobjectfollowingacertainschema:

//storeschemainvariabletomakeitre-usable,see.body()

constdocSchema=joi.object().required().keys({

name:joi.string().required(),

age:joi.number().required()

}).unknown();//allowadditionalattributes

router.post('/entries',function(req,res){

constmultiple=Array.isArray(req.body);

constbody=multiple?req.body:[req.body];

letdata=[];

for(vardocofbody){

constmeta=foxxColl.save(doc);

data.push(Object.assign(doc,meta));

res.send(multiple?data:data[0]);

.body(joi.alternatives().try(

docSchema,

joi.array().items(docSchema)

),'Entryorentriestostoreinthecollection.')

.response(joi.alternatives().try(

joi.object().required(),

joi.array().items(joi.object().required())

),'Entryorentriesstoredinthecollection.')

.summary('Storeentryorentries')

.description('Storeasingleentryormultipleentriesinthe"myFoxxCollection"collection.');

WritingdatabasequeriesStoringandretrievingentriesisfine,butrightnowwehavetomemorizeeachkeywhenwecreateanentry.Let'saddaroutethatgivesusalistofthekeysofallentriessowecanusethosetolookanentryupindetail.

ThenaïveapproachwouldbetousethetoArray()methodtoconverttheentirecollectiontoanarrayandjustreturnthat.Butwe'reonlyinterestedinthekeysandtheremightpotentiallybesomanyentriesthatfirstretrievingeverysingledocumentmightgetunwieldy.Let'swriteashortAQLquerytodothisinstead:

//continued

constaql=require('@arangodb').aql;

router.get('/entries',function(req,res){

constkeys=db._query(aql`

FORentryIN${foxxColl}

RETURNentry._key

res.send(keys);

.response(joi.array().items(

joi.string().required()

Gettingstarted

).required(),'Listofentrykeys.')

.summary('Listentrykeys')

.description('Assemblesalistofkeysofentriesinthecollection.');

Herewe'reusingtwonewthings:

The_querymethodexecutesanAQLqueryintheactivedatabase.

Theaqltemplatestringhandlerallowsustowritemulti-lineAQLqueriesandalsohandlesqueryparametersandcollectionnames.InsteadofhardcodingthenameofthecollectionwewanttouseinthequerywecansimplyreferencethefoxxCollvariablewedefinedearlier--itrecognizesthevalueasanArangoDBcollectionobjectandknowswearespecifyingacollectionratherthanaregularvalueeventhoughAQLdistinguishesbetweenthetwo.

Note:Ifyouaren'tusedtoJavaScripttemplatestringsandtemplatestringhandlersjustthinkofaqlasafunctionthatreceivesthemultilinestringsplitatevery${}expressionaswellasanarrayofthevaluesofthoseexpressions--that'sactuallyallthereistoit.

Alternatively,here'saversionwithouttemplatestrings(noticehowmuchcleanertheaqlversionwillbeincomparisonwhenyouhavemultiplevariables):

constkeys=db._query(

'FORentryIN@@collRETURNentry._key',

{'@coll':foxxColl.name()}

Nextsteps

YounowknowhowtocreateaFoxxservicefromscratch,howtohandleuserinputandhowtoaccessthedatabasefromwithinyourFoxxservicetostore,retrieveandquerydatayoustoreinsideArangoDB.ThisshouldallowyoutobuildmeaningfulAPIsforyourownapplicationsbuttherearemanymorethingsyoucandowithFoxx:

Needtogofaster?Turnondevelopmentmodeandhackonyourcoderightontheserver.

Concernedaboutsecurity?Youcouldaddauthenticationtoyourservicetoprotectaccesstothedatabeforeitevenleavesthedatabase.

Writingasinglepageapp?YoucouldstoresomebasicassetsrightinsideyourFoxxservice.

Needtointegrateexternalservices?YoucanmakeHTTPrequestsfrominsideFoxxandusequeuedjobstoperformthatworkinthebackground.

Tiredofreinventingthewheel?Learnaboutdependencies.

Everythingbroken?Youcanwriteteststomakesureyourlogicremainssound.

Gettingstarted

ManifestfilesEveryservicecomeswithamanifest.jsonfileprovidingmetadata.Thefollowingfieldsareallowedinmanifests:

configuration:Object(optional)

Anobjectdefiningtheconfigurationoptionsthisservicerequires.

defaultDocument:string(optional)

Ifspecified,the/(root)routeoftheservicewillautomaticallyredirecttothegivenrelativepath,e.g.:

"defaultDocument":"index.html"

ThiswouldhavethesameeffectascreatingthefollowingrouteinJavaScript:

constindexRouter=createRouter();

indexRouter.all('/',function(req,res){

res.redirect('index.html');

module.context.use(indexRouter);

Note:Asof3.0.0thisfieldcansafelybeomitted;thevaluenolongerdefaultsto"index.html".

dependencies:Object(optional)andprovides:Object(optional)

Objectsspecifyingotherservicesthisservicehasasdependenciesandwhatdependenciesitcanprovidetootherservices.

engines:Object(optional)

AnobjectindicatingthesemanticversionrangesofArangoDB(orcompatibleenvironments)theservicewillbecompatiblewith,e.g.:

"engines":{

"arangodb":"^3.0.0"

ThisshouldcorrectlyindicatetheminimumversionofArangoDBtheservicehasbeentestedagainst.FoxxmaintainsastrictsemanticversioningpolicyasofArangoDB3.0.0soitisgenerallysafetousesemverranges(e.g. 3.0.0tomatchanyversiongreaterorequalto3.0.0andbelow4.0.0)formaximumcompatibility.

files:Object(optional)

Anobjectdefiningfileassetsservedbythisservice.

lib:string(Default:".")

TherelativepathtotheFoxxJavaScriptfilesintheservice,e.g.:

"lib":"lib"

Thiswouldresultinthemainentrypoint(seebelow)andotherJavaScriptpathsbeingresolvedasrelativetothelibfolderinsidetheservicefolder.

main:string(optional)

Therelativepathtothemainentrypointofthisservice(relativetolib,seeabove),e.g.:

"main":"index.js"

ThiswouldresultinFoxxloadingandexecutingthefileindex.jswhentheserviceismountedorstarted.

Servicemanifest

Note:whileitistechnicallypossibletoomitthisfield,youwilllikelywanttoprovideanentrypointtoyourserviceasthisistheonlywaytoexposeHTTProutesorexportaJavaScriptAPI.

scripts:Object(optional)

Anobjectdefiningnamedscriptsprovidedbythisservice,whichcaneitherbeuseddirectlyorasqueuedjobsbyotherservices.

tests:stringorArray<string>(optional)

ApathorlistofpathsofJavaScripttestsprovidedforthisservice.

Additionallymanifestscanprovidethefollowingmetadata:

author:string(optional)

Thefullnameoftheauthoroftheservice(i.e.you).Thiswillbeshowninthewebinterface.

contributors:Array<string>(optional)

Alistofnamesofpeoplethathavecontributedtothedevelopmentoftheserviceinsomeway.Thiswillbeshowninthewebinterface.

description:string(optional)

Ahuman-readabledescriptionoftheservice.Thiswillbeshowninthewebinterface.

keywords:Array<string>(optional)

Alistofkeywordsthathelpcategorizethisservice.ThisisusedbytheFoxxStoreinstallerstoorganizeservices.

license:string(optional)

Astringidentifyingthelicenseunderwhichtheserviceispublished,ideallyintheformofanSPDXlicenseidentifier.Thiswillbeshowninthewebinterface.

name:string(optional)

ThenameoftheFoxxservice.AllowedcharactersareA-Z,0-9,theASCIIhyphen(-)andunderscore(_)characters.Thenamemustnotstartwithanumber.Thiswillbeshowninthewebinterface.

thumbnail:string(optional)

Thefilenameofathumbnailthatwillbeusedalongsidetheserviceinthewebinterface.ThisshouldbeaJPEGorPNGimagethatlooksgoodatsizes50x50and160x160.

version:string(optional)

TheversionnumberoftheFoxxservice.Theversionnumbermustfollowthesemanticversioningformat.Thiswillbeshowninthewebinterface.

Examples

"name":"example-foxx-service",

"version":"3.0.0-dev",

"license":"MIT",

"description":"Anexampleservicewitharelativelyfull-featuredmanifest.",

"thumbnail":"foxx-icon.png",

"keywords":["demo","service"],

"author":"ArangoDBGmbH",

"contributors":[

"AlanPlum<alan@arangodb.example>"

"lib":"dist",

"main":"entry.js",

"defaultDocument":"welcome.html",

"engines":{

"arangodb":"^3.0.0"

"files":{

Servicemanifest

"welcome.html":"assets/index.html",

"hello.jpg":"assets/hello.jpg"

"world.jpg":{

"path":"assets/world.jpg",

"type":"image/jpeg",

"gzip":false

"tests":"dist/**.spec.js"

Servicemanifest

FoxxservicecontextTheservicecontextprovidesaccesstomethodsandattributesthatarespecifictoagivenservice.InaFoxxservicethecontextisgenerallyavailableasthemodule.contextvariable.Withinarouter'srequesthandlertherequestandresponseobjects'contextattributealsoprovideaccesstothecontextoftheservicetheroutewasmountedin(whichmaybedifferentfromtheonetheroutehandlerwasdefinedin).

Examples

//inservice/my-foxx-1

//Seethechapterondependenciesformoreinfoon

//howexportsanddependenciesworkacrossservices

module.exports={routes:router};

router.get(function(req,res){

module.context.mount==='/my-foxx-1';

req.context.mount==='/my-foxx-2';

res.write('Hellofrommy-foxx-1');

//inservice/my-foxx-2

constrouter2=createRouter();

module.context.use(router2);

router2.post(function(req,res){

module.context.mount==='/my-foxx-2';

req.context.mount==='/my-foxx-2';

res.write('Hellofrommy-foxx-2');

constrouter1=module.context.dependencies.myFoxx1.routes;

module.context.use(router1);

Theservicecontextspecifiesthefollowingproperties:

argv:any

Anyargumentspassedinifthecurrentfilewasexecutedasascriptorqueuedjob.

basePath:string

Thefilesystempathoftheservice,i.e.thefolderinwhichtheservicewasinstalledtobyArangoDB.

baseUrl:string

ThebaseURLoftheservice,relativetotheArangoDBserver,e.g./_db/_system/my-foxx.

collectionPrefix:string

TheprefixthatwillbeusedbycollectionandcollectionNametoderivethenamesofservice-specificcollections.Thisisderivedfromtheservice'smountpoint,e.g./my-foxxbecomesmy_foxx.

configuration:Object

Configurationoptionsfortheservice.

dependencies:Object

Configureddependenciesfortheservice.

isDevelopment:boolean

Indicateswhethertheserviceisrunningindevelopmentmode.

Servicecontext

isProduction:boolean

TheinverseofisDevelopment.

manifest:Object

Theparsedmanifestfileoftheservice.

mount:string

Themountpointoftheservice,e.g./my-foxx.

apiDocumentationmodule.context.apiDocumentation([options]):Function

DEPRECATED

CreatesarequesthandlerthatservestheAPIdocumentation.

Note:ThismethodhasbeendeprecatedinArangoDB3.1andreplacedwiththemorestraightforwardcreateDocumentationRoutermethodprovidingthesamefunctionality.

Arguments

SeecreateDocumentationRouterbelow.

Examples

//ServetheAPIdocsforthecurrentservice

router.get('/docs/*',module.context.apiDocumentation());

//Notethatthepathmustendwithawildcard

//andtheroutemustuseHTTPGET.

createDocumentationRoutermodule.context.createDocumentationRouter([options]):Router

CreatesarouterthatservestheAPIdocumentation.

Note:Theroutercanbemountedlikeanyotherchildrouter(seeexamplesbelow).

Arguments

options:Object(optional)

Anobjectwithanyofthefollowingproperties:

mount:string(Default:module.context.mount)

Themountpathoftheservicetoservethedocumentationof.

indexFile:string(Default:"index.html")

FilenameoftheHTMLfileservingtheAPIdocumentation.

swaggerRoot:string(optional)

FullpathofthefoldercontainingtheSwaggerassetsandtheindexFile.DefaultstotheSwaggerassetsusedbythewebinterface.

before:Function(optional)

Afunctionthatwillbeexecutedbeforearequestishandled.

Ifthefunctionreturnsfalsetherequestwillnotbeprocessedanyfurther.

Ifthefunctionreturnsanobject,itsattributeswillbeusedtooverridetheoptionsforthecurrentrequest.

Servicecontext

Anyotherreturnvaluewillbeignored.

Ifoptionsisafunctionitwillbeusedasthebeforeoption.

IfoptionsisastringitwillbeusedastheswaggerRootoption.

ReturnsaFoxxrouter.

Examples

//ServetheAPIdocsforthecurrentservice

router.use('/docs',module.context.createDocumentationRouter());

//--or--

//ServetheAPIdocsfortheservicetherouterismountedin

router.use('/docs',module.context.createDocumentationRouter(function(req){

return{mount:req.context.mount};

//--or--

//ServetheAPIdocsonlyforusersauthenticatedwithArangoDB

router.use('/docs',module.context.createDocumentationRouter(function(req,res){

if(req.suffix==='swagger.json'&&!req.arangoUser){

res.throw(401,'Notauthenticated');

collectionmodule.context.collection(name):ArangoCollection|null

PassesthegivennametocollectionName,thenlooksupthecollectionwiththeprefixedname.

Arguments

name:string

Unprefixednameoftheservice-specificcollection.

Returnsacollectionornullifnocollectionwiththeprefixednameexists.

collectionNamemodule.context.collectionName(name):string

PrefixesthegivennamewiththecollectionPrefixforthisservice.

Arguments

name:string

Unprefixednameoftheservice-specificcollection.

Returnstheprefixedname.

Examples

module.context.mount==='/my-foxx'

module.context.collectionName('doodads')==='my_foxx_doodads'

filemodule.context.file(name,[encoding]):Buffer|string

PassesthegivennametofileName,thenloadsthefilewiththeresultingname.

Servicecontext

Arguments

name:string

Nameofthefiletoload,relativetothecurrentservice.

encoding:string(optional)

Encodingofthefile,e.g.utf-8.Ifomittedthefilewillbeloadedasarawbufferinsteadofastring.

Returnsthefile'scontents.

fileNamemodule.context.fileName(name):string

Resolvesthegivenfilenamerelativetothecurrentservice.

Arguments

name:string

Nameofthefile,relativetothecurrentservice.

Returnstheabsolutefilepath.

usemodule.context.use([path],router):Endpoint

Mountsagivenrouterontheservicetoexposetherouter'sroutesontheservice'smountpoint.

Arguments

path:string(Default:"/")

Pathtomounttherouterat,relativetotheservice'smountpoint.

router:Router|Middleware

Arouterormiddlewaretomount.

ReturnsanEndpointforthegivenrouterormiddleware.

Note:Mountingservicesatruntime(e.g.withinrequesthandlersorqueuedjobs)isnotsupported.

Servicecontext

FoxxconfigurationFoxxservicescandefineconfigurationparameterstomakethemmorere-usable.

Theconfigurationobjectmapsnamestoconfigurationparameters:

Thekeyisthenameunderwhichtheparameterwillbeavailableontheservicecontext'sconfigurationproperty.

Thevalueisaparameterdefinition.

Thekeyshouldbeavalididentifierfollowingthecase-insensitiveformat/^[_$a-z][-_$a-z0-9]*$/.

Theparameterdefinitioncanhavethefollowingproperties:

description:string

Humanreadabledescriptionoftheparameter.

type:string(Default:"string")

Typeoftheconfigurationparameter.Supportedvaluesare:

"integer"or"int":anyfiniteintegernumber.

"boolean"or"bool":thevaluestrueorfalse.

"number":anyfinitedecimalorintegernumber.

"string":anystringvalue.

"json":anywell-formedJSONvalue.

"password":likestringbutwillbedisplayedasamaskedinputfieldinthewebfrontend.

default:any

Defaultvalueoftheconfigurationparameter.

required:(Default:true)

Whethertheparameterisrequired.

Iftheconfigurationhasparametersthatdonotspecifyadefaultvalue,youneedtoconfiguretheservicebeforeitbecomesactive.InthemeantimeafallbackservicelicationwillbemountedthatrespondstoallrequestswithaHTTP500statuscodeindicatingaserver-sideerror.

TheconfigurationparametersofamountedservicecanbeadjustedfromthewebinterfacebyclickingtheConfigurationbuttonintheservicedetails.

Examples

"configuration":{

"currency":{

"description":"Currencysymboltouseforpricesintheshop.",

"default":"$",

"type":"string"

"secretKey":{

"description":"Secretkeytouseforsigningsessiontokens.",

"type":"password"

Configuration

DependencymanagementTherearetwothingscommonlycalled"dependencies"inFoxx:

Moduledependencies,i.e.dependenciesonexternalJavaScriptmodules(e.g.fromthepublicnpmregistry)

Foxxdependencies,i.e.dependenciesbetweenFoxxservices

Let'slookattheminmoredetail:

Moduledependencies

Youcanusethenode_modulesfoldertobundlethird-partyFoxx-compatiblenpmandNode.jsmoduleswithyourFoxxservice.Typicallythisisachievedbyaddingapackage.jsonfiletoyourprojectspecifyingnpmdependenciesusingthedependenciesattributeandinstallingthemwiththenpmcommand-linetool.

Makesuretoincludetheactualnode_modulesfolderinyourFoxxservicebundleasArangoDBwillnotdoanythingspecialtoinstallthesedependencies.AlsokeepinmindthatbundlingextraneousmoduleslikedevelopmentdependenciesmaybloatthefilesizeofyourFoxxservicebundle.

Compatibilitycaveats

UnlikeJavaScriptinbrowsersorNode.js,theJavaScriptenvironmentinArangoDBissynchronous.ThismeansanymodulesthatdependonasynchronousbehaviourlikepromisesorsetTimeoutwillnotbehavecorrectlyinArangoDBorFoxx.AdditionallyunlikeNode.jsArangoDBdoesnotsupportnativeextensions.AllmoduleshavetobeimplementedinpureJavaScript.

WhileArangoDBprovidesalotofcompatibilitycodetosupportmoduleswrittenforNode.js,someNode.jsbuilt-inmodulescannotbeprovidedbyArangoDB.ForacloserlookattheNode.jsmodulesArangoDBdoesordoesnotprovidecheckouttheappendixonJavaScriptmodules.

Alsonotethattheserestrictionsnotonlyapplyonthemodulesyouwishtoinstallbutalsothedependenciesofthosemodules.Asaruleofthumb:moduleswrittentoworkinNode.jsandthebrowserthatdonotrelyonasyncbehaviourshouldgenerallywork;modulesthatrelyonnetworkorfilesystemI/Oormakeheavyuseofasyncbehaviourmostlikelywillnot.

Foxxdependencies

Foxxdependenciescanbedeclaredinaservice'smanifestusingtheprovidesanddependenciesfields:

providesliststhedependenciesagivenserviceprovides,i.e.whichAPIsitclaimstobecompatiblewith

dependenciesliststhedependenciesagivenserviceuses,i.e.whichAPIsitsdependenciesneedtobecompatiblewith

Adependencynameshouldgenerallyusethesameformatasanamespaced(org-scoped)NPMmodule,e.g.@foxx/sessions.Thedependencynameshouldfolloweitherthecase-insensitiveformat/^@[_$a-z][-_$a-z0-9]*(\/[_$a-z][-_$a-z0-9]*)*$/or/^[_$a-z][-_$a-z0-9]*$/.

DependencynamesrefertotheexternalJavaScriptAPIofaserviceratherthanspecificservicesimplementingthoseAPIs.Somedependencynamesdefinedbyofficiallymaintainedservicesare:

@foxx/auth(version1.0.0)@foxx/api-keys(version1.0.0)@foxx/bugsnag(versions1.0.0and2.0.0)@foxx/mailgun(versions1.0.0and2.0.0)@foxx/postageapp(versions1.0.0and2.0.0)@foxx/postmark(versions1.0.0and2.0.0)@foxx/sendgrid(versions1.0.0and2.0.0)@foxx/oauth2(versions1.0.0and2.0.0)@foxx/segment-io(versions1.0.0and2.0.0)

Dependencies

@foxx/sessions(versions1.0.0and2.0.0)@foxx/users(versions1.0.0,2.0.0and3.0.0)

Aprovidesdefinitionmapseachprovideddependency'snametotheprovidedversion:

"provides":{

"@foxx/auth":"1.0.0"

Adependenciesdefinitionmapsthelocalaliasofagivendependencyagainstitsnameandthesupportedversionrange(eitherasaJSONobjectorashorthandstring):

"dependencies":{

"mySessions":"@foxx/sessions:^2.0.0",

"myAuth":{

"name":"@foxx/auth",

"version":"^1.0.0",

"description":"Thisdescriptionisentirelyoptional.",

"required":false

Dependenciescanbeconfiguredfromthewebinterfaceinaservice'ssettingstabusingtheDependenciesbutton.

Thelocalaliasshouldbeavalididentifierfollowingthecase-insensitiveformat/^[_$a-z][-_$a-z0-9]*$/.

Thevalueforeachdependencyshouldbethedatabase-relativemountpathoftheservice(includingtheleadingslash).Inordertobeusableasthedependencyofanotherservicebothservicesneedtobemountedinthesamedatabase.Aservicecanbeusedtoprovidemultipledependenciesforthesameservice(aslongastheexpectedJavaScriptAPIsdon'tconflict).

Aservicethathasunconfiguredrequireddependenciescannotbeuseduntilallofitsdependencieshavebeenconfigured.

Itispossibletospecifythemountpathofaservicethatdoesnotactuallydeclarethedependencyasprovided.Thereiscurrentlynovalidationbeyondthemanifestformats.

Whenaserviceusesanothermountedserviceasadependencythedependency'smainentryfile'sexportsobjectbecomesavailableinthemodule.context.dependenciesobjectoftheotherservice:

Examples

ServiceAandServiceBaremountedinthesamedatabase.ServiceBhasadependencywiththelocalalias"greeter".ThedependencyisconfiguredtousethemountpathofServiceA.

//EntryfileofServiceA

module.exports={

sayHi(){

return'Hello';

//SomewhereinServiceB

constgreeter=module.context.dependencies.greeter;

res.write(greeter.sayHi());

Dependencies

RoutersconstcreateRouter=require('@arangodb/foxx/router');

RoutersletyoudefineroutesthatextendArangoDB'sHTTPAPIwithcustomendpoints.

RoutersneedtobemountedusingtheusemethodofaservicecontexttoexposetheirHTTProutesataservice'smountpath.

Youcanpassroutersbetweenservicesmountedinthesamedatabaseasdependencies.Youcanevennestrouterswithineachother.

CreatingaroutercreateRouter():Router

Thisreturnsanew,cleanrouterobjectthathasnotyetbeenmountedintheserviceandcanbeexportedlikeanyotherobject.

Requesthandlersrouter.get([path],[...middleware],handler,[name]):Endpoint

router.post([path],[...middleware],handler,[name]):Endpoint

router.put([path],[...middleware],handler,[name]):Endpoint

router.patch([path],[...middleware],handler,[name]):Endpoint

router.delete([path],[...middleware],handler,[name]):Endpoint

router.all([path],[...middleware],handler,[name]):Endpoint

Thesemethodsletyouspecifyroutesontherouter.TheallmethoddefinesaroutethatwillmatchanysupportedHTTPverb,theothermethodsdefineroutesthatonlymatchtheHTTPverbwiththesamename.

Arguments

path:string(Default:"/")

ThepathoftherequesthandlerrelativetothebasepaththeRouterismountedat.Ifomitted,therequesthandlerwillhandlerequeststothebasepathoftheRouter.Forinformationondefiningdynamicroutesseethesectiononpathparametersinthechapteronrouterendpoints.

middleware:Function(optional)

Zeroormoremiddlewarefunctionsthattakethefollowingarguments:

req:Request

Anincomingserverrequestobject.

res:Response

Anoutgoingserverresponseobject.

next:Function

Acallbackthatpassescontrolovertothenextmiddlewarefunctionandreturnswhenthatfunctionhascompleted.

Ifatruthyargumentispassed,thatargumentwillbethrownasanerror.

Ifthereisnonextmiddlewarefunction,thehandlerwillbeinvokedinstead(seebelow).

handler:Function

Afunctionthattakesthefollowingarguments:

req:Request

Anincomingserverrequestobject.

Routers

res:Response

Anoutgoingserverresponse.

AnamethatcanbeusedtogenerateURLsfortheendpoint.Formoreinformationseethereversemethodoftherequestobject.

ReturnsanEndpointfortheroute.

Examples

Simpleindexroute:

res.set('content-type','text/plain');

res.write('HelloWorld!');

RestrictingaccesstoauthenticatedArangoDBusers:

router.get('/secrets',function(req,res,next){

if(req.arangoUser){

next();

}else{

res.throw(404,'Secrets?Whatsecrets?');

},function(req,res){

res.download('allOurSecrets.zip');

Multiplemiddlewarefunctions:

functioncounting(req,res,next){

if(!req.counter)req.counter=0;

req.counter++;

next();

req.counter--;

router.get(counting,counting,counting,function(req,res){

res.json({counter:req.counter});//{"counter":3}

Mountingchildroutersandmiddlewarerouter.use([path],middleware,[name]):Endpoint

Theusemethodletsyoumountachildrouterormiddlewareatagivenpath.

Arguments

path:string(optional)

ThepathofthemiddlewarerelativetothebasepaththeRouterismountedat.Ifomitted,themiddlewarewillhandlerequeststothebasepathoftheRouter.Forinformationondefiningdynamicroutesseethesectiononpathparametersinthechapteronrouterendpoints.

middleware:Router|Middleware

Anunmountedrouterobjectoramiddleware.

AnamethatcanbeusedtogenerateURLsforendpointsofthisrouter.Formoreinformationseethereversemethodoftherequestobject.HasnoeffectifhandlerisaMiddleware.

ReturnsanEndpointforthemiddlewareorchildrouter.

Routers

EndpointsEndpointsarereturnedbytheuse,allandHTTPverb(e.g.get,post)methodsofroutersaswellastheusemethodoftheservicecontext.Theycanbeusedtoattachmetadatatomountedroutes,middlewareandchildroutersthataffectshowrequestsandresponsesareprocessedorprovidesAPIdocumentation.

Endpointsshouldonlybeusedtoinvokethefollowingmethods.Endpointmethodscanbechainedtogether(eachmethodreturnstheendpointitself).

headerendpoint.header(name,[schema],[description]):this

Definesarequestheaderrecognizedbytheendpoint.Anyadditionalnon-definedheaderswillbetreatedasoptionalstringvalues.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.

Iftheendpointisachildrouter,allroutesofthatrouterwillusethisheaderdefinitionunlessoverridden.

Arguments

name:string

Nameoftheheader.Thisshouldbeconsideredcaseinsensitiveasallheadernameswillbeconvertedtolowercase.

schema:Schema(optional)

Aschemadescribingtheformatoftheheadervalue.Thiscanbeajoischemaoranythingthathasacompatiblevalidatemethod.

Thevalueofthisheaderwillbesettothevaluepropertyofthevalidationresult.Avalidationfailurewillresultinanautomatic400(BadRequest)errorresponse.

AhumanreadablestringthatwillbeshownintheAPIdocumentation.

Returnstheendpoint.

Examples

router.get(/*...*/)

.header('arangoVersion',joi.number().min(30000).default(30000));

pathParamendpoint.pathParam(name,[schema],[description]):this

Definesapathparameterrecognizedbytheendpoint.Pathparametersareexpectedtobefilledaspartoftheendpoint'smountpath.Anyadditionalnon-definedpathparameterswillbetreatedasoptionalstringvalues.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.

Iftheendpointisachildrouter,allroutesofthatrouterwillusethisparameterdefinitionunlessoverridden.

Arguments

name:string

Nameoftheparameter.

Aschemadescribingtheformatoftheparameter.Thiscanbeajoischemaoranythingthathasacompatiblevalidatemethod.

Endpoints

Thevalueofthisparameterwillbesettothevaluepropertyofthevalidationresult.Avalidationfailurewillresultintheroutefailingtomatchandbeingignored(resultingina404(NotFound)errorresponseifnootherroutesmatch).

Returnstheendpoint.

Examples

router.get('/some/:num/here',/*...*/)

.pathParam('num',joi.number().required());

queryParamendpoint.queryParam(name,[schema],[description]):this

Definesaqueryparameterrecognizedbytheendpoint.Anyadditionalnon-definedqueryparameterswillbetreatedasoptionalstringvalues.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.

Iftheendpointisachildrouter,allroutesofthatrouterwillusethisparameterdefinitionunlessoverridden.

Arguments

name:string

Nameoftheparameter.

Aschemadescribingtheformatoftheparameter.Thiscanbeajoischemaoranythingthathasacompatiblevalidatemethod.

Thevalueofthisparameterwillbesettothevaluepropertyofthevalidationresult.Avalidationfailurewillresultinanautomatic400(BadRequest)errorresponse.

Returnstheendpoint.

Examples

router.get(/*...*/)

.queryParam('num',joi.number().required());

bodyendpoint.body([model],[mimes],[description]):this

Definestherequestbodyrecognizedbytheendpoint.Therecanonlybeonerequestbodydefinitionperendpoint.ThedefinitionwillalsobeshownintheroutedetailsintheAPIdocumentation.

Intheabsenceofarequestbodydefinition,therequestobject'sbodypropertywillbeinitializedtotheunprocessedrawBodybuffer.

Iftheendpointisachildrouter,allroutesofthatrouterwillusethisbodydefinitionunlessoverridden.Iftheendpointisamiddleware,therequestbodywillonlybeparsedonce(i.e.theMIMEtypesoftheroutematchingthesamerequestwillbeignoredbutthebodywillstillbevalidatedagain).

Arguments

model:Model|Schema|null(optional)

Amodelorjoischemadescribingtherequestbody.Avalidationfailurewillresultinanautomatic400(BadRequest)errorresponse.

Endpoints

IfthevalueisamodelwithafromClientmethod,thatmethodwillbeappliedtotheparsedrequestbody.

Ifthevalueisaschemaoramodelwithaschema,theschemawillbeusedtovalidatetherequestbodyandthevaluepropertyofthevalidationresultoftheparsedrequestbodywillbeusedinsteadoftheparsedrequestbodyitself.

IfthevalueisamodeloraschemaandtheMIMEtypehasbeenomitted,theMIMEtypewilldefaulttoJSONinstead.

Ifthevalueisexplicitlysettonull,norequestbodywillbeexpected.

Ifthevalueisanarraycontainingexactlyonemodelorschema,therequestbodywillbetreatedasanarrayofitemsmatchingthatmodelorschema.

mimes:Array<string>(optional)

AnarrayofMIMEtypestheroutesupports.

Commonnon-mimealiaseslike"json"or"html"arealsosupportedandwillbeexpandedtotheappropriateMIMEtype(e.g."application/json"and"text/html").

IftheMIMEtypeisrecognizedbyFoxxtherequestbodywillbeparsedintotheappropriatestructurebeforebeingvalidated.CurrentlyonlyJSON,application/x-www-form-urlencodedandmultipartformatsaresupportedinthisway.

IftheMIMEtypeindicatedintherequestheadersdoesnotmatchanyofthesupportedMIMEtypes,thefirstMIMEtypeinthelistwillbeusedinstead.

Failuretoparsetherequestbodywillresultinanautomatic400(BadRequest)errorresponse.

Returnstheendpoint.

Examples

router.post('/expects/some/json',/*...*/)

.body(

joi.object().required(),

'ThisimpliesJSON.'

router.post('/expects/nothing',/*...*/)

.body(null);//Nobodyallowed

router.post('/expects/some/plaintext',/*...*/)

.body(['text/plain'],'Thisbodywillbeastring.');

responseendpoint.response([status],[model],[mimes],[description]):this

Definesaresponsebodyfortheendpoint.Whenusingtheresponseobject'ssendmethodintherequesthandlerofthisroute,thedefinitionwiththematchingstatuscodewillbeusedtogeneratetheresponsebody.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.

Iftheendpointisachildrouter,allroutesofthatrouterwillusethisresponsedefinitionunlessoverridden.Iftheendpointisamiddleware,thismethodhasnoeffect.

Arguments

status:number|string(Default:200or204)

HTTPstatuscodetheresponseappliesto.Ifastringisprovidedinsteadofanumericstatuscodeitwillbeusedtolookupanumericstatuscodeusingthestatusesmodule.

model:Model|Schema|null(optional)

Amodelorjoischemadescribingtheresponsebody.

Endpoints

IfthevalueisamodelwithaforClientmethod,thatmethodwillbeappliedtothedatapassedtoresponse.sendwithintherouteiftheresponsestatuscodematches(butalsoifnostatuscodehasbeenset).

Ifthevalueisaschemaoramodelwithaschema,theactualschemawillnotbeusedtovalidatetheresponsebodyandonlyservestodocumenttheresponseinmoredetailintheAPIdocumentation.

IfthevalueisamodeloraschemaandtheMIMEtypehasbeenomitted,theMIMEtypewilldefaulttoJSONinstead.

Ifthevalueisexplicitlysettonullandthestatuscodehasbeenomitted,thestatuscodewilldefaultto204("nocontent")insteadof200.

Ifthevalueisanarraycontainingexactlyonemodelorschema,theresponsebodywillbeanarrayofitemsmatchingthatmodelorschema.

mimes:Array<string>(optional)

AnarrayofMIMEtypestheroutemightrespondwithforthisstatuscode.

Commonnon-mimealiaseslike"json"or"html"arealsosupportedandwillbeexpandedtotheappropriateMIMEtype(e.g."application/json"and"text/html").

Whenusingtheresponse.sendmethodtheresponsebodywillbeconvertedtotheappropriateMIMEtypeifpossible.

Ahuman-readablestringthatbrieflydescribestheresponseandwillbeshownintheendpoint'sdetaileddocumentation.

Returnstheendpoint.

Examples

//Thisexampleonlyprovidesdocumentation

//andimpliesagenericJSONresponsebody.

router.get(/*...*/)

.response(

joi.array().items(joi.string()),

'Alistofdoodadidentifiers.'

//Noresponsebodywillbeexpectedhere.

router.delete(/*...*/)

.response(null,'Thedoodadnolongerexists.');

//Anendpointcandefinemultipleresponsetypes

//fordifferentstatuscodes--butnevermorethan

//oneforeachstatuscode.

router.post(/*...*/)

.response('found','Thedoodadislocatedelsewhere.')

.response(201,['text/plain'],'Thedoodadwascreatedsohereisahaiku.');

//Heretheresponsebodywillbesetto

//thequerystring-encodedresultof

//FormModel.forClient({some:'data'})

//becausethestatuscodedefaultsto200.

router.patch(function(req,res){

res.send({some:'data'});

.response(FormModel,['application/x-www-form-urlencoded'],'OMG.');

//Inthiscasetheresponsebodywillbesetto

//SomeModel.forClient({some:'data'})because

//thestatuscodehasbeensetto201before.

router.put(function(req,res){

res.status(201);

res.send({some:'data'});

.response(201,SomeModel,'Somethingamazinghappened.');

Endpoints

endpoint.error(status,[description]):this

Documentsanerrorstatusfortheendpoint.

Iftheendpointisachildrouter,allroutesofthatrouterwillusethiserrordescriptionunlessoverridden.Iftheendpointisamiddleware,thismethodhasnoeffect.

ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.

Arguments

status:number|string

HTTPstatuscodefortheerror(e.g.400for"badrequest").Ifastringisprovidedinsteadofanumericstatuscodeitwillbeusedtolookupanumericstatuscodeusingthestatusesmodule.

Ahuman-readablestringthatbrieflydescribestheerrorconditionandwillbeshownintheendpoint'sdetaileddocumentation.

Returnstheendpoint.

Examples

res.throw(403,'Validationerroratx.y.z');

.error(403,'Indicatesthatavalidationhasfailed.');

summaryendpoint.summary(summary):this

Addsashortdescriptiontotheendpoint'sAPIdocumentation.

Iftheendpointisachildrouter,allroutesofthatrouterwillusethissummaryunlessoverridden.Iftheendpointisamiddleware,thismethodhasnoeffect.

Arguments

summary:string

Ahuman-readablestringthatbrieflydescribestheendpointandwillappearnexttotheendpoint'spathinthedocumentation.

Returnstheendpoint.

Examples

router.get(/*...*/)

.summary('Listalldiscombobulateddoodads')

descriptionendpoint.description(description):this

Addsalongdescriptiontotheendpoint'sAPIdocumentation.

Iftheendpointisachildrouter,allroutesofthatrouterwillusethisdescriptionunlessoverridden.Iftheendpointisamiddleware,thismethodhasnoeffect.

Arguments

Endpoints

description:string

Ahuman-readablestringthatdescribestheendpointindetailandwillbeshownintheendpoint'sdetaileddocumentation.

Returnstheendpoint.

Examples

//The"dedent"libraryhelpsformatting

//multi-linestringsbyadjustingindentation

//andremovingleadingandtrailingblanklines

constdd=require('dedent');

router.post(/*...*/)

.description(dd`

Thisroutediscombobulatesthedoodadsby

frobnicatingthemoxieoftherequestbody.

deprecatedendpoint.deprecated([deprecated]):this

Markstheendpointasdeprecated.

Iftheendpointisachildrouter,allroutesofthatrouterwillalsobemarkedasdeprecated.Iftheendpointisamiddleware,thismethodhasnoeffect.

Arguments

deprecated:boolean(Default:true)

Whethertheendpointshouldbemarkedasdeprecated.Ifsettofalsetheendpointwillbeexplicitlymarkedasnotdeprecated.

Returnstheendpoint.

Examples

router.get(/*...*/)

.deprecated();

tagendpoint.tag(...tags):this

MarkstheendpointwiththegiventagsthatwillbeusedtogrouprelatedroutesinthegeneratedAPIdocumentation.

Iftheendpointisachildrouter,allroutesofthatrouterwillalsobemarkedwiththetags.Iftheendpointisamiddleware,thismethodhasnoeffect.

Arguments

tags:string

Oneormorestringsthatwillbeusedtogrouptheendpoint'sroutes.

Returnstheendpoint.

Examples

router.get(/*...*/)

.tag('auth','restricted');

Endpoints

MiddlewareMiddlewareinFoxxreferstofunctionsthatcanbemountedlikeroutesandcanmanipulatetherequestandresponseobjectsbeforeandaftertherouteitselfisinvoked.Theycanalsobeusedtocontrolaccessortoprovidecommonlogiclikeloggingetc.Unlikeroutes,middlewareismountedwiththeusemethodlikearouter.

Insteadofafunctiontheusemethodcanalsoacceptanobjectwitharegisterfunctionthatwilltakeaparameterendpoint,themiddlewarewillbemountedatandreturnstheactualmiddlewarefunction.Thisallowsmanipulatingtheendpointbeforecreatingthemiddleware(e.g.todocumentheaders,requestbodies,pathparametersorqueryparameters).

Examples

RestrictaccesstoArangoDB-authenticatedusers:

module.context.use(function(req,res,next){

if(!req.arangoUser){

res.throw(401,'NotauthenticatedwithArangoDB');

next();

Anytruthyargumentpassedtothenextfunctionwillbethrownasanerror:

leterr=null;

err=newError('Thisshouldneverhappen');

next(err);//throwsiftheerrorwasset

Trivialloggingmiddleware:

conststart=Date.now();

next();

}finally{

console.log(`Handledrequestin${Date.now()-start}ms`);

Morecomplexexampleforheader-basedsessions:

constsessions=module.context.collection('sessions');

module.context.use({

register(endpoint){

endpoint.header('x-session-id',joi.string().optional(),'ThesessionID.');

returnfunction(req,res,next){

constsid=req.get('x-session-id');

if(sid){

req.session=sessions.document(sid);

}catch(e){

deletereq.headers['x-session-id'];

next();

if(req.session){

if(req.session._rev){

sessions.replace(req.session,req.session);

res.set('x-session-id',req.session._key);

}else{

constmeta=sessions.save(req.session);

res.set('x-session-id',meta._key);

Middleware

RequestobjectsTherequestobjectspecifiesthefollowingproperties:

arangoUser:string|null

TheauthenticatedArangoDBusernameusedtomaketherequest.ThisvalueisonlysetifauthenticationisenabledinArangoDBandtherequestsetanauthorizationheaderArangoDBwasabletoverify.YouarestronglyencouragedtoimplementyourownauthenticationlogicforyourownservicesbutthispropertycanbeusefulifyouneedtointegratewithArangoDB'sownauthenticationmechanisms.

arangoVersion:number

Thenumericvalueofthex-arango-versionheaderorthenumericversionoftheArangoDBserver(e.g.30102forversion3.1.2)ifnovalidheaderwasprovided.

baseUrl:string

Root-relativebaseURLoftheservice,i.e.theprefix"/_db/"followedbythevalueofdatabase.

body:any

Theprocessedandvalidatedrequestbodyforthecurrentroute.Ifnobodyhasbeendefinedforthecurrentroute,thevaluewillbeidenticaltorawBody.

FordetailsonhowrequestbodiescanbeprocessedandvalidatedbyFoxxseethebodymethodoftheendpointobject.

context:Context

Theservicecontextinwhichtherouterwasmounted(ratherthanthecontextinwhichtheroutewasdefined).

database:string

Thenameofthedatabaseinwhichtherequestisbeinghandled,e.g."_system".

headers:object

Therawheadersobject.

FordetailsonhowrequestheaderscanbevalidatedbyFoxxseetheheadermethodoftheendpointobject.

hostname:string

Thehostname(domainname)indicatedintherequestheaders.

Defaultstothehostnameportion(i.e.excludingtheport)oftheHostheaderandfallsbacktothelisteningaddressoftheserver.

method:string

TheHTTPverbusedtomaketherequest,e.g."GET".

originalUrl:string

Root-relativeURLoftherequest,i.e.pathfollowedbytherawqueryparameters,ifany.

path:string

Database-relativepathoftherequestURL(notincludingthequeryparameters).

pathParams:object

Anobjectmappingthenamesofpathparametersofthecurrentroutetotheirvalidatedvalues.

FordetailsonhowpathparameterscanbevalidatedbyFoxxseethepathParammethodoftheendpointobject.

port:number

Theportindicatedintherequestheaders.

Request

Defaultstotheportportion(i.e.excludingthehostname)oftheHostheaderandfallsbacktothelisteningportortheappropriatedefaultport(443forHTTPSor80forHTTP,dependingonsecure)iftheheaderonlyindicatesahostname.

Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettotheportportionoftheX-Forwarded-Hostheader(orapproriatedefaultport)ifpresent.

protocol:string

Theprotocolusedfortherequest.

Defaultsto"https"or"http"dependingonwhetherArangoDBisconfiguredtouseSSLornot.

Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothevalueoftheX-Forwarded-Protoheaderifpresent.

queryParams:object

Anobjectmappingthenamesofqueryparametersofthecurrentroutetotheirvalidatedvalues.

FordetailsonhowqueryparameterscanbevalidatedbyFoxxseethequeryParammethodoftheendpointobject.

rawBody:Buffer

Theraw,unparsed,unvalidatedrequestbodyasabuffer.

remoteAddress:string

TheIPoftheclientthatmadetherequest.

Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothefirstIPlistedintheX-Forwarded-Forheaderifpresent.

remoteAddresses:Array<string>

AlistcontainingtheIPaddressesusedtomaketherequest.

DefaultstothevalueofremoteAddresswrappedinanarray.

Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothelistofIPsspecifiedintheX-Forwarded-Forheaderifpresent.

remotePort:number

Thelisteningportoftheclientthatmadetherequest.

Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettotheportspecifiedintheX-Forwarded-Portheaderifpresent.

secure:boolean

Whethertherequestwasmadeoverasecureconnection(i.e.HTTPS).

Thisissettofalsewhenprotocolis"http"andtruewhenprotocolis"https".

suffix:string

Thetrailingpathrelativetothecurrentrouteifthecurrentrouteendsinawildcard(e.g./something/*).

Note:StartingwithArangoDB3.2ispassedintotheserviceas-is,i.e.percentageescapesequenceslike%2Fwillnolongerbeunescaped.Alsonotethatthesuffixmaycontainpathsegmentslike..whichmayhavespecialmeaningifthesuffixisusedtobuildfilesystempaths.

trustProxy:boolean

Indicateswhethertherequestwasmadeusingatrustedproxy.Iftheoriginserver'saddresswasspecifiedintheArangoDBconfigurationusing--frontend.trusted-proxyortheservice'strustProxysettingisenabled,thiswillbetrue,otherwiseitwillbefalse.

url:string

TheURLoftherequest.

Request

xhr:boolean

WhethertherequestindicatesitwasmadewithinabrowserusingAJAX.

ThisissettotrueiftheX-Requested-Withheaderispresentandisacase-insensitivematchforthevalue"xmlhttprequest".

NotethatthisvaluedoesnotguaranteewhethertherequestwasmadefrominsideabrowserorwhetherAJAXwasusedandismerelyaconventionestablishedbyJavaScriptframeworkslikejQuery.

acceptsreq.accepts(types):string|false

req.accepts(...types):string|false

req.acceptsCharsets(charsets):string|false

req.acceptsCharsets(...charsets):string|false

req.acceptsEncodings(encodings):string|false

req.acceptsEncodings(...encodings):string|false

req.acceptsLanguages(languages):string|false

req.acceptsLanguages(...languages):string|false

Thesemethodswrapthecorrespondingcontentnegotiationmethodsoftheacceptsmoduleforthecurrentrequest.

Examples

if(req.accepts(['json','html'])==='html'){

//ClientexplicitlyprefersHTMLoverJSON

res.write('<h1>ClientprefersHTML</h1>');

}else{

//OtherwisejustsendJSON

res.json({success:true});

cookiereq.cookie(name,options):string|null

Getsthevalueofacookiebyname.

Arguments

name:string

Nameofthecookie.

options:object(optional)

secret:string(optional)

Secretthatwasusedtosignthecookie.

Ifasecretisspecified,thecookie'ssignatureisexpectedtobepresentinasecondcookiewiththesamenameandthesuffix.sig.Otherwisethesignature(ifpresent)willbeignored.

algorithm:string(Default:"sha256")

Algorithmthatwasusedtosignthecookie.

Ifastringispassedinsteadofanoptionsobjectitwillbeinterpretedasthesecretoption.

Returnsthevalueofthecookieornullifthecookieisnotsetoritssignatureisinvalid.

Request

get/headerreq.get(name):string

req.header(name):string

Getsthevalueofaheaderbyname.Youcanvalidaterequestheadersusingtheheadermethodoftheendpoint.

Arguments

name:string

Nameoftheheader.

Returnstheheadervalue.

isreq.is(types):string

req.is(...types):string

Thismethodwrapsthe(requestbody)contenttypedetectionmethodofthetype-ismoduleforthecurrentrequest.

Examples

consttype=req.is('html','application/xml','application/*+xml');

if(type===false){//nomatch

handleDefault(req.rawBody);

}elseif(type==='html'){

handleHtml(req.rawBody);

}else{//isXML

handleXml(req.rawBody);

jsonreq.json():any

AttemptstoparsetherawrequestbodyasJSONandreturnstheresult.

Itisgenerallymoreusefultodefinearequestbodyontheendpointandusethereq.bodypropertyinstead.

Returnsundefinediftherequestbodyisempty.MaythrowaSyntaxErrorifthebodycouldnotbeparsed.

makeAbsolutereq.makeAbsolute(path,[query]):string

Resolvesthegivenpathrelativetothereq.context.service'smountpathtoafullURL.

Arguments

path:string

Thepathtoresovle.

query:string|object

AstringorobjectwithqueryparameterstoaddtotheURL.

ReturnstheformattedabsoluteURL.

paramsreq.param(name):any

Request

Arguments

Looksupaparameterbyname,preferringpathParamsoverqueryParams.

It'sprobablybetterstyletousethereq.pathParamsorreq.queryParamsobjectsdirectly.

name:string

Nameoftheparameter.

Returnsthe(validated)valueoftheparameter.

rangereq.range([size]):Ranges|number

Thismethodwrapstherangeheaderparsingmethodoftherange-parsermoduleforthecurrentrequest.

Arguments

size:number(Default:Infinity)

Lengthofthesatisfiablerange(e.g.numberofbytesinthefullresponse).Ifpresent,rangesexceedingthesizewillbeconsideredunsatisfiable.

ReturnsundefinediftheRangeheaderisabsent,-2iftheheaderispresentbutmalformed,-1iftherangeisinvalid(e.g.startoffsetislargerthanendoffset)orunsatisfiableforthegivensize.

Otherwisereturnsanarrayofobjectswiththepropertiesstartandendvaluesforeachrange.Thearrayhasanadditionalpropertytypeindicatingtherequestrangetype.

Examples

console.log(req.headers.range);//"bytes=40-80"

constranges=req.range(100);

console.log(ranges);//[{start:40,end:80}]

console.log(ranges.type);//"bytes"

reversereq.reverse(name,[params]):string

LooksuptheURLofanamedrouteforthegivenparameters.

Arguments

name:string

Nameoftheroutetolookup.

params:object(optional)

Anobjectcontainingvaluesforthe(pathorquery)parametersoftheroute.

ReturnstheURLoftherouteforthegivenparameters.

Examples

router.get('/items/:id',function(req,res){

/*...*/

},'getItemById');

router.post('/items',function(req,res){

consturl=req.reverse('getItemById',{id:createdItem._key});

res.set('location',req.makeAbsolute(url));

Request

ResponseobjectsTheresponseobjectspecifiesthefollowingproperties:

body:Buffer|string

Responsebodyasastringorbuffer.Canbesetdirectlyorusingsomeoftheresponsemethods.

context:Context

Theservicecontextinwhichtherouterwasmounted(ratherthanthecontextinwhichtheroutewasdefined).

headers:object

Therawheadersobject.

statusCode:number

Statuscodeoftheresponse.Defaultsto200(bodysetandnotanemptystringorbuffer)or204(otherwise)ifnotchangedfromundefined.

attachmentres.attachment([filename]):this

Setsthecontent-dispositionheadertoindicatetheresponseisadownloadablefilewiththegivenname.

Note:Thisdoesnotactuallymodifytheresponsebodyoraccessthefilesystem.TosendafilefromthefilesystemseethedownloadorsendFilemethods.

Arguments

filename:string(optional)

Nameofthedownloadablefileintheresponsebody.

Ifpresent,theextensionofthefilenamewillbeusedtosettheresponsecontent-typeifithasnotyetbeenset.

Returnstheresponseobject.

cookieres.cookie(name,value,[options]):this

Setsacookiewiththegivenname.

Arguments

name:string

Nameofthecookie.

value:string

Valueofthecookie.

ttl:number(optional)

Timetoliveofthecookieinseconds.

algorithm:string(Default:"sha256")

Algorithmthatwillbeusedtosignthecookie.

Response

Secretthatwillbeusedtosignthecookie.

Ifasecretisspecified,thecookie'ssignaturewillbestoredinasecondcookiewiththesameoptions,thesamenameandthesuffix.sig.Otherwisenosignaturewillbeadded.

Pathforwhichthecookieshouldbeissued.

domain:string(optional)

Domainforwhichthecookieshouldbeissued.

secure:boolean(Default:false)

Whetherthecookieshouldbemarkedassecure(i.e.HTTPS/SSL-only).

httpOnly:boolean(Default:false)

WhetherthecookieshouldbemarkedasHTTP-only(ratherthanalsoexposingittoclient-sidecode).

Ifanumberispassedinsteadofanoptionsobjectitwillbeinterpretedasthettloption.

downloadres.download(path,[filename]):this

Theequivalentofcallingres.attachment(filename).sendFile(path).

Arguments

path:string

Pathtothefileonthelocalfilesystemtobesentastheresponsebody.

filename:string(optional)

Filenametoindicateinthecontent-dispositionheader.

Ifomittedthepathwillbeusedinstead.

getHeaderres.getHeader(name):string

Getsthevalueoftheheaderwiththegivenname.

Arguments

name:string

Nameoftheheadertoget.

Returnsthevalueoftheheaderorundefined.

jsonres.json(data):this

SetstheresponsebodytotheJSONstringvalueofthegivendata.

Response

Arguments

data:any

Thedatatobeusedastheresponsebody.

redirectres.redirect([status],path):this

Redirectstheresponsebysettingtheresponselocationheaderandstatuscode.

Arguments

status:number|string(optional)

Responsestatuscodetoset.

Ifthestatuscodeisthestringvalue"permanent"itwillbetreatedasthevalue301.

Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.

Ifthestatuscodeisomittedbuttheresponsestatushasnotalreadybeenset,theresponsestatuswillbesetto302.

path:string

URLtosetthelocationheaderto.

removeHeaderres.removeHeader(name):this

Removestheheaderwiththegivennamefromtheresponse.

Arguments

name:string

Nameoftheheadertoremove.

sendres.send(data,[type]):this

Setstheresponsebodytothegivendatawithrespecttotheresponsedefinitionfortheresponse'scurrentstatuscode.

Arguments

data:any

Thedatatobeusedastheresponsebody.Willbeconvertedaccordingtheresponsedefinitionfortheresponse'scurrentstatuscode(or200)inthefollowingway:

IfthedataisanArangoDBresultset,itwillbeconvertedtoanarrayfirst.

IftheresponsedefinitionspecifiesamodelwithaforClientmethod,thatmethodwillbeappliedtothedatafirst.Ifthedataisanarrayandtheresponsedefinitionhasthemultipleflagset,themethodwillbeappliedtoeachentryindividuallyinstead.

Finallythedatawillbeprocessedbytheresponsetypehandlertoconvertheresponsebodytoastringorbuffer.

type:string(Default:"auto")

Response

Content-typeoftheresponsebody.

Ifsetto"auto"thefirstMIMEtypespecifiedintheresponsedefinitionfortheresponse'scurrentstatuscode(or200)willbeusedinstead.

Ifsetto"auto"andnoresponsedefinitionexists,theMIMEtypewillbedeterminedthefollowingway:

IfthedataisabuffertheMIMEtypewillbesettobinary(application/octet-stream).

IfthedataisanobjecttheMIMEtypewillbesettoJSONandthedatawillbeconvertedtoaJSONstring.

OtherwisetheMIMEtypewillbesettoHTMLandthedatawillbeconvertedtoastring.

sendFileres.sendFile(path,[options]):this

Sendsafilefromthelocalfilesystemastheresponsebody.

Arguments

path:string

Pathtothefileonthelocalfilesystemtobesentastheresponsebody.

Ifnocontent-typeheaderhasbeensetyet,theextensionofthefilenamewillbeusedtosetthevalueofthatheader.

lastModified:boolean(optional)

Ifsettotrueorifnolast-modifiedheaderhasbeensetyetandthevalueisnotsettofalsethelast-modifiedheaderwillbesettothemodificationdateofthefileinmilliseconds.

Examples

//Sendthefile"favicon.ico"fromthisservice'sfolder

res.sendFile(module.context.fileName('favicon.ico'));

sendStatusres.sendStatus(status):this

Sendsaplaintextresponseforthegivenstatuscode.Theresponsestatuswillbesettothegivenstatuscode,theresponsebodywillbesettothestatusmessagecorrespondingtothatstatuscode.

Arguments

setHeader/setres.setHeader(name,value):this

res.set(name,value):this

Response

res.set(headers):this

Setsthevalueoftheheaderwiththegivenname.

Arguments

name:string

Nameoftheheadertoset.

value:string

Valuetosettheheaderto.

headers:object

Headerobjectmappingheadernamestovalues.

statusres.status(status):this

Setstheresponsestatustothegivenstatuscode.

Arguments

throwres.throw(status,[reason],[options]):void

ThrowsanHTTPexceptionforthegivenstatus,whichwillbehandledbyFoxxtoservetheappropriateJSONerrorresponse.

Arguments

Ifthestatuscodeisinthe500-range(500-599),itsstacktracewillalwaysbeloggedasifitwereanunhandledexception.

Ifdevelopmentmodeisenabled,theerror'sstacktracewillbeloggedasawarningifthestatuscodeisinthe400-range(400-499)orasaregularmessageotherwise.

reason:string(optional)

Messagefortheexception.

Ifomitted,thestatusmessagecorrespondingtothestatuscodewillbeusedinstead.

cause:Error(optional)

Causeoftheexceptionthatwillbeloggedaspartoftheerror'sstacktrace(recursively,iftheexceptionalsohasacausepropertyandsoon).

Response

extra:object(optional)

AdditionalpropertiesthatwillbeaddedtotheerrorresponsebodygeneratedbyFoxx.

Ifdevelopmentmodeisenabled,anexceptionpropertywillbeaddedtothisvaluecontainingtheerrormessageandastacktracepropertywillbeaddedcontaininganarraywitheachlineoftheerror'sstacktrace.

Ifanerrorispassedinsteadofanoptionsobjectitwillbeinterpretedasthecauseoption.Ifnoreasonwasprovidedtheerror'smessagewillbeusedasthereasoninstead.

Returnsnothing.

typeres.type([type]):string

Setstheresponsecontent-typetothegiventypeifprovidedorreturnsthepreviouslysetcontent-type.

Arguments

type:string(optional)

Content-typeoftheresponsebody.

Unlikeres.set('content-type',type)fileextensionscanbeprovidedasvaluesandwillbetranslatedtothecorrespondingMIMEtype(e.g.jsonbecomesapplication/json).

Returnsthecontent-typeoftheresponsebody.

varyres.vary(names):this

res.vary(...names):this

Thismethodwrapsthevaryheadermanipulationmethodofthevarymoduleforthecurrentresponse.

Thegivennameswillbeaddedtotheresponse'svaryheaderifnotalreadypresent.

Examples

res.vary('user-agent');

res.vary('cookie');

res.vary('cookie');//duplicateswillbeignored

//--or--

res.vary('user-agent','cookie');

//--or--

res.vary(['user-agent','cookie']);

writeres.write(data):this

Appendsthegivendatatotheresponsebody.

Arguments

data:string|Buffer

Datatoappend.

Response

Ifthedataisabuffertheresponsebodywillbeconvertedtoabufferfirst.

Iftheresponsebodyisabufferthedatawillbeconvertedtoabufferfirst.

IfthedataisanobjectitwillbeconvertedtoaJSONstringfirst.

Ifthedataisanyothernon-stringvalueitwillbeconvertedtoastringfirst.

Response

UsingGraphQLinFoxxconstcreateGraphQLRouter=require('@arangodb/foxx/graphql');

Foxxbundlesthegraphql-syncmodule,whichisasynchronouswrapperfortheofficialJavaScriptGraphQLreferenceimplementation,toallowwritingGraphQLschemasdirectlyinsideFoxx.

Additionallythe@arangodb/foxx/graphqlletsyoucreateroutersforservingGraphQLrequests,whichcloselymimicksthebehaviouroftheexpress-graphqlmodule.

Formoreinformationongraphql-syncseethegraphql-jsAPIreference(notethatgraphql-syncalwaysusesrawvaluesinsteadofwrappingtheminpromises).

ForanexampleofaGraphQLschemainFoxxthatresolvesfieldsusingthedatabaseseetheGraphQLexampleservice(alsoavailablefromtheFoxxstore).

Examples

constgraphql=require('graphql-sync');

constgraphqlSchema=newgraphql.GraphQLSchema({

//Mountingagraphqlendpointdirectlyinaservice:

module.context.use('/graphql',createGraphQLRouter({

schema:graphqlSchema,

graphiql:true

//Orattheservice'srootURL:

module.context.use(createGraphQLRouter({

graphiql:true

//Orinsideanexistingrouter:

router.get('/hello',function(req,res){

res.write('Helloworld!');

router.use('/graphql',createGraphQLRouter({

graphiql:true

Note:ArangoDBaimsforstabilitywhichmeansbundleddependencieswillgenerallynotbeupdatedasquicklyastheirmaintainersmakeupdatesavailableonGitHuborNPM.StartingwithArangoDB3.2,ifyouwanttouseanewerreleaseofgraphql-syncthantheonebundledwithyourtargetversionofArangoDB,youcanprovideyourownversionofthelibrarybypassingitviathegraphqloption:

constgraphql=require('graphql-sync');

constgraphqlSchema=newgraphql.Schema({

module.context.use(createGraphQLRouter({

graphiql:true,

graphql:graphql

ThismakessureFoxxusesthegraphql-syncmodulebundledinyourservice'snode_modulesfolder(ifavailable)insteadofthebuilt-inversion.Ifyoufindgraphql-syncitselflaggingbehindtheofficialgraphqlmodule,consideropeninganissueonitsGitHubrepository.

CreatingaroutercreateGraphQLRouter(options):Router

UsingGraphQL

ThisreturnsanewrouterobjectwithPOSTandGETroutesforservingGraphQLrequests.

Arguments

options:object

schema:GraphQLSchema

AGraphQLSchemaobjectfromgraphql-sync.

context:any(optional)

TheGraphQLcontextthatwillbepassedtothegraphql()functionfromgraphql-synctohandleGraphQLqueries.

rootValue:object(optional)

TheGraphQLrootvaluethatwillbepassedtothegraphql()functionfromgraphql-synctohandleGraphQLqueries.

pretty:boolean(Default:false)

Iftrue,JSONresponseswillbepretty-printed.

formatError:Function(optional)

Afunctionthatwillbeusedtoformaterrorsproducedbygraphql-sync.Ifomitted,theformatErrorfunctionfromgraphql-syncwillbeusedinstead.

validationRules:Array<any>(optional)

AdditionalvalidationrulesqueriesmustsatisfyinadditiontothosedefinedintheGraphQLspec.

graphiql:boolean(Default:false)

Iftrue,theGraphiQLexplorerwillbeservedwhenloadeddirectlyfromabrowser.

graphql:object(optional)

Ifyouneedtouseyourowncopyofthegraphql-syncmoduleinsteadoftheonebundledwithArangoDB,hereyoucanpassitindirectly.

IfaGraphQLSchemaobjectispassedinsteadofanoptionsobjectitwillbeinterpretedastheschemaoption.

Generatedroutes

TherouterhandlesGETandPOSTrequeststoitsrootpathandacceptsthefollowingparameters,whichcanbeprovidedeitherasqueryparametersorasthePOSTrequestbody:

query:string

AGraphQLquerythatwillbeexecuted.

variables:object|string(optional)

AnobjectorastringcontainingaJSONobjectwithruntimevaluestouseforanyGraphQLqueryvariables.

operationName:string(optional)

Iftheprovidedquerycontainsmultiplenamedoperations,thisspecifieswhichoperationshouldbeexecuted.

raw:boolean(Default:false)

ForcesaJSONresponseevenifgraphiqlisenabledandtherequestwasmadeusingabrowser.

ThePOSTrequestbodycanbeprovidedasJSONorasquerystringusingapplication/x-www-form-urlencoded.Arequestbodypassedasapplication/graphqlwillbeinterpretedasthequeryparameter.

UsingGraphQL

SessionMiddlewareconstsessionMiddleware=require('@arangodb/foxx/sessions');

ThesessionmiddlewareaddsthesessionandsessionStoragepropertiestotherequestobjectanddealswithserializinganddeserializingthesessionaswellasextractingsessionidentifiersfromincomingrequestsandinjectingthemintooutgoingresponses.

Examples

//Createasessionmiddleware

constsessions=sessionsMiddleware({

storage:module.context.collection('sessions'),

transport:['header','cookie']

//Firstenablethemiddlewareforthisservice

module.context.use(sessions);

//Nowmounttheroutersthatusethesession

router.get('/',function(req,res){

res.send(`Hello${req.session.uid||'anonymous'}!`);

},'hello');

router.post('/login',function(req,res){

req.session.uid=req.body;

req.sessionStorage.save(req.session);

res.redirect(req.reverse('hello'));

.body(['text/plain'],'Username');

CreatingasessionmiddlewaresessionMiddleware(options):Middleware

Createsasessionmiddleware.

Arguments

options:Object

Anobjectwiththefollowingproperties:

storage:Storage

Storagethatwillbeusedtopersistthesessions.

ThestorageisalsoexposedasthesessionStorageonallrequestobjectsandasthestoragepropertyofthemiddleware.

IfastringorcollectionispassedinsteadofaStorage,itwillbeusedtocreateaCollectionStorage.

transport:Transport|Array<Transport>

Transportorarrayoftransportsthatwillbeusedtoextractthesessionidentifiersfromincomingrequestsandinjectthemintooutgoingresponses.Whenattemptingtoextractasessionidentifier,thetransportswillbeusedintheorderspecifieduntilamatchisfound.Wheninjecting(orclearing)sessionidentifiers,alltransportswillbeinvoked.

Thetransportsarealsoexposedasthetransportpropertyofthemiddleware.

Ifthestring"cookie"ispassedinsteadofaTransport,theCookieTransportwillbeusedwiththedefaultsettingsinstead.

Ifthestring"header"ispassedinsteadofaTransport,theHeaderTransportwillbeusedwiththedefaultsettingsinstead.

autoCreate:boolean(Default:true)

Ifenabledthesessionstorage'snewmethodwillbeinvokedtocreateanemptysessionwheneverthetransportfailedtoreturnasessionfortheincomingrequest.Otherwisethesessionwillbeinitializedasnull.

Sessionsmiddleware

Returnsthesessionmiddleware.

Sessionsmiddleware

SessionStoragesSessionstoragesareusedbythesessionsmiddlewaretopersistsessionsacrossrequests.SessionstoragesmustimplementthefromClientandforClientmethodsandcanoptionallyimplementthenewmethod.

Thebuilt-insessionstoragesgenerallyprovidethefollowingattributes:

uid:string(Default:null)

Auniqueidentifierindicatingtheactiveuser.

created:number(Default:Date.now())

Thenumerictimestampofwhenthesessionwascreated.

data:any(Default:null)

Arbitrarydatatopersistedinthesession.

newstorage.new():Session

Generatesanewsessionobjectrepresentinganemptysession.Theemptysessionobjectshouldnotbepersistedunlessnecessary.Thereturnvaluewillbeexposedbythemiddlewareasthesessionpropertyoftherequestobjectifnosessionidentifierwasreturnedbythesessiontransportsandauto-creationisnotexplicitlydisabledinthesessionmiddleware.

Examples

new(){

return{

uid:null,

created:Date.now(),

data:null

fromClientstorage.fromClient(sid):Session|null

Resolvesordeserializesasessionidentifiertoasessionobject.

Arguments

sid:string

Sessionidentifiertoresolveordeserialize.

Returnsasessionobjectrepresentingthesessionwiththegivensessionidentifierthatwillbeexposedbythemiddlewareasthesessionpropertyoftherequestobject.Thismethodwillonlybecalledifanyofthesessiontransportsreturnedasessionidentifier.Ifthesessionidentifierisinvalidorexpired,themethodshouldreturnanullvaluetoindicatenomatchingsession.

Examples

fromClient(sid){

returndb._collection('sessions').firstExample({_key:sid});

forClientstorage.forClient(session):string|null

Sessionstorages

Derivesasessionidentifierfromthegivensessionobject.

Arguments

session:Session

Sessiontoderiveasessionidentifierfrom.

Returnsasessionidentifierforthesessionrepresentedbythegivensessionobject.Thismethodwillbecalledwiththesessionpropertyoftherequestobjectunlessthatpropertyisempty(e.g.null).

Examples

forClient(session){

if(!session._key){

constmeta=db._collection('sessions').save(session);

returnmeta._key;

db._collection('sessions').replace(session._key,session);

returnsession._key;

Sessionstorages

CollectionSessionStorageconstcollectionStorage=require('@arangodb/foxx/sessions/storages/collection');

Thecollectionsessionstoragepersistssessionstoacollectioninthedatabase.

CreatingastoragecollectionStorage(options):Storage

CreatesaStoragethatcanbeusedinthesessionsmiddleware.

Arguments

options:Object

collection:ArangoCollection

Thecollectionthatshouldbeusedtopersistthesessions.Ifastringispassedinsteadofacollectionitisassumedtobethefullyqualifiednameofacollectioninthecurrentdatabase.

ttl:number(Default:60*60)

Thetimeinsecondssincethelastupdateuntilasessionwillbeconsideredexpired.

pruneExpired:boolean(Default:false)

Whetherexpiredsessionsshouldberemovedfromthecollectionwhentheyareaccessedinsteadofsimplybeingignored.

autoUpdate:boolean(Default:true)

Whethersessionsshouldbeupdatedinthecollectioneverytimetheyareaccessedtokeepthemfromexpiring.Disablingthisoptionwillimproveperformancebutmeansyouwillhavetotakecareofkeepingyoursessionsaliveyourself.

Ifastringorcollectionispassedinsteadofanoptionsobject,itwillbeinterpretedasthecollectionoption.

prunestorage.prune():Array<string>

Removesallexpiredsessionsfromthecollection.ThismethodshouldbecalledevenifthepruneExpiredoptionisenabledtocleanupabandonedsessions.

Returnsanarrayofthekeysofallsessionsthatwereremoved.

savestorage.save(session):Session

Saves(replaces)thegivensessionobjectinthecollection.Thismethodneedstobeinvokedexplicitlyaftermakingchangestothesessionorthechangeswillnotbepersisted.Assignsanew_keytothesessionifitpreviouslydidnothaveone.

Arguments

session:Session

Asessionobject.

Returnsthemodifiedsession.

Sessionstorages

storage.clear(session):boolean

Removesthesessionfromthecollection.Hasnoeffectifthesessionwasalreadyremovedorhasnotyetbeensavedtothecollection(i.e.hasno_key).

Arguments

session:Session

Asessionobject.

Returnstrueifthesessionwasremovedorfalseifithadnoeffect.

Sessionstorages

JWTSessionStorageconstjwtStorage=require('@arangodb/foxx/sessions/storages/jwt');

TheJWTsessionstorageconvertssessionstoandfromJSONWebTokens.

Examples

//PassinasecuresecretfromtheFoxxconfiguration

constsecret=module.context.configuration.jwtSecret;

storage:jwtStorage(secret),

transport:'header'

CreatingastoragejwtStorage(options):Storage

CreatesaStoragethatcanbeusedinthesessionsmiddleware.

Note:whilethe"none"algorithm(i.e.nosignature)issupportedthisdummyalgorithmprovidesnosecurityandallowsclientstomakearbitrarymodificationstothepayloadandshouldnotbeusedunlessyouarecertainyouspecificallyneedit.

Arguments

options:Object

algorithm:string(Default:"HS512")

Thealgorithmtouseforsigningthetoken.

Supportedvalues:

"HS256"(HMAC-SHA256)"HS384"(HMAC-SHA384)"HS512"(HMAC-SHA512)"none"(nosignature)

secret:string

Thesecrettouseforsigningthetoken.

Thisfieldisforbiddenwhenusingthe"none"algorithmbutrequiredotherwise.

ttl:number(Default:3600)

Themaximumlifetimeofthetokeninseconds.Youmaywanttokeepthisshortasanewtokenisgeneratedoneveryrequestallowingclientstorefreshtokensautomatically.

verify:boolean(Default:true)

Ifsettofalsethesignaturewillnotbeverifiedbutstillgenerated(unlessusingthe"none"algorithm).

maxExp:number(Default:Infinity)

LargestvaluethatwillbeacceptedinanincomingJWTexp(expiration)field.

Sessionstorages

SessionTransportsSessiontransportsareusedbythesessionsmiddlewaretostoreandretrievesessionidentifiersinrequestsandresponses.Sessiontransportsmustimplementthegetand/orsetmethodsandcanoptionallyimplementtheclearmethod.

gettransport.get(request):string|null

Retrievesasessionidentifierfromarequestobject.

Ifpresentthismethodwillautomaticallybeinvokedforeachtransportuntilatransportreturnsasessionidentifier.

Arguments

request:Request

Requestobjecttoextractasessionidentifierfrom.

Returnsthesessionidentifierornullifthetransportcannotfindasessionidentifierintherequest.

Examples

get(req){

returnreq.get('x-session-id')||null;

settransport.set(response,sid):void

Attachesasessionidentifiertoaresponseobject.

Ifpresentthismethodwillautomaticallybeinvokedattheendofarequestregardlessofwhetherthesessionwasmodifiedornot.

Arguments

response:Response

Responseobjecttoattachasessionidentifierto.

sid:string

Sessionidentifiertoattachtotheresponse.

Returnsnothing.

Examples

set(res){

res.set('x-session-id',value);

cleartransport.clear(response):void

Attachesapayloadindicatingthatthesessionhasbeenclearedtotheresponseobject.Thiscanbeusedtoclearasessioncookiewhenthesessionhasbeendestroyed(e.g.duringlogout).

Ifpresentthismethodwillautomaticallybeinvokedinsteadofsetwhenthereq.sessionattributewasremovedbytheroutehandler.

Arguments

Sessiontransports

response:Response

Responseobjecttoremovethesessionidentifierfrom.

Returnsnothing.

Sessiontransports

CookieSessionTransportconstcookieTransport=require('@arangodb/foxx/sessions/transports/cookie');

Thecookietransportstoressessionidentifiersincookiesontherequestandresponseobject.

Examples

//PassinasecuresecretfromtheFoxxconfiguration

constsecret=module.context.configuration.cookieSecret;

transport:cookieTransport({

name:'FOXXSESSID',

ttl:60*60*24*7,//oneweekinseconds

algorithm:'sha256',

secret:secret

CreatingatransportcookieTransport([options]):Transport

CreatesaTransportthatcanbeusedinthesessionsmiddleware.

Arguments

name:string(Default:"sid")

Thenameofthecookie.

ttl:number(optional)

Cookielifetimeinseconds.NotethatthisdoesnotaffectthestorageTTL(i.e.howlongthesessionitselfisconsideredvalid),justhowlongthecookieshouldbestoredbytheclient.

algorithm:string(optional)

Thealgorithmusedtosignandverifythecookie.Ifnoalgorithmisspecified,thecookiewillnotbesignedorverified.Seethecookiemethodontheresponseobject.

Secrettouseforthesignedcookie.Willbeignoredifnoalgorithmisprovided.

Pathforwhichthecookieshouldbeissued.

domain:string(optional)

Domainforwhichthecookieshouldbeissued.

secure:boolean(Default:false)

Whetherthecookieshouldbemarkedassecure(i.e.HTTPS/SSL-only).

httpOnly:boolean(Default:false)

WhetherthecookieshouldbemarkedasHTTP-only(ratherthanalsoexposingittoclient-sidecode).

Ifastringispassedinsteadofanoptionsobject,itwillbeinterpretedasthenameoption.

Sessiontransports

HeaderSessionTransportconstheaderTransport=require('@arangodb/foxx/sessions/transports/header');

Theheadertransportstoressessionidentifiersinheadersontherequestandresponseobjects.

Examples

transport:headerTransport('X-FOXXSESSID')

CreatingatransportheaderTransport([options]):Transport

CreatesaTransportthatcanbeusedinthesessionsmiddleware.

Arguments

name:string(Default:X-Session-Id)

Nameoftheheaderthatcontainsthesessionidentifier(notcasesensitive).

Ifastringispassedinsteadofanoptionsobject,itwillbeinterpretedasthenameoption.

Sessiontransports

StaticfileassetsThemostflexiblewaytoservefilesinyourFoxxserviceistosimplypassthemthroughinyourrouterusingthecontextobject'sfileNamemethodandtheresponseobject'ssendFilemethod:

router.get('/some/filename.png',function(req,res){

constfilePath=module.context.fileName('some-local-filename.png');

res.sendFile(filePath);

Whileallowingforgreatercontrolofhowthefileshouldbesenttotheclientandwhoshouldbeabletoaccessit,doingthisforallyourstaticassetscangettedious.

AlternativelyyoucanspecifyfileassetsthatshouldbeservedbyyourFoxxservicedirectlyintheservicemanifestusingthefilesattribute:

"files":{

"/some/filename.png":{

"path":"some-local-filename.png",

"type":"image/png",

"gzip":false

"/favicon.ico":"bookmark.ico",

"/static":"my-assets-folder"

Eachentryinthefilesattributecanrepresenteitherasinglefileoradirectory.Whenservingentiredirectories,thekeyactsasaprefixandrequeststothatprefixwillberesolvedwithinthegivendirectory.

Options

path:string

Therelativepathofthefileorfolderwithintheservice.

type:string(optional)

TheMIMEcontenttypeofthefile.Defaultstoanintelligentguessbasedonthefilename'sextension.

gzip:boolean(Default:false)

Ifsettotruethefilewillbeservedwithgzip-encodingifsupportedbytheclient.Thiscanbeusefulwhenservingtextfileslikeclient-sideJavaScript,CSSorHTML.

Ifastringisprovidedinsteadofanobject,itwillbeinterpretedasthepathoption.

Servingfiles

WritingtestsFoxxprovidesoutoftheboxsupportforrunningtestsagainstaninstalledserviceusingtheMochatestrunner.

TestfileshavefullaccesstotheservicecontextandallArangoDBAPIsbutlikescriptscannotdefineFoxxroutes.

Runningtests

Aninstalledservice'stestscanbeexecutedfromtheadministrativewebinterface:

1. Openthe"Services"tabofthewebinterface2. Clickontheinstalledservicetobetested3. Clickonthe"Settings"tab4. Clickontheflaskiconinthetopright5. Accepttheconfirmationdialog

Notethatrunningtestsinaproductiondatabaseisnotrecommendedandmayresultindatalossifthetestsaccessthedatabase.

Whenrunningaserviceindevelopmentmodespecialcareneedstobetakenasperformingrequeststotheservice'sownroutesaspartofthetestsuitesmayresultintestsbeingexecutedwhilethedatabaseisinaninconsistentstate,leadingtounexpectedbehaviour.

TestfilepathsInordertotellFoxxaboutfilescontainingtestsuites,oneormorepatternsneedtobespecifiedinthetestsoptionoftheservicemanifest:

"tests":[

"**/test_*.js",

"**/*_test.js"

Thesepatternscanbeeitherrelativefilepathsor"globstar"patternswhere

*matcheszeroormorecharactersinafilename**matcheszeroormorenesteddirectories

Forexample,giventhefollowingdirectorystructure:

++test/

||+-a1.js

||+-a2.js

||+-test.js

|+-b.js

|+-c.js

|+-d_test.js

+-e_test.js

+-test.js

Thefollowingpatternswouldmatchthefollowingfiles:

test.js:

test.js

test/*.js:

/test/b.js

/test/c.js

/test/d_test.js

Writingtests

test/**/*.js:

/test/a/a1.js

/test/a/a2.js

/test/a/test.js

/test/b.js

/test/c.js

/test/d_test.js

**/test.js:

/test/a/test.js

**/*test.js:

/test/a/test.js

/test/d_test.js

/e_test.js

/test.js

Evenifmultiplepatternsmatchthesamefilethetestsinthatfilewillonlyberunonce.

Theorderoftestsisalwaysdeterminedbythefilepaths,nottheorderinwhichtheyarematchedorspecifiedinthemanifest.

TeststructureMochatestsuitescanbedefinedusingoneofthreeinterfaces:BDD,TDDorExports.

TheQUnitinterfaceofMochaisnotsupportedinArangoDB.

LikeallArangoDBcode,testcodeisalwayssynchronous.

BDDinterface

TheBDDinterfacedefinestestsuitesusingthedescribefunctionandeachtestcaseisdefinedusingtheitfunction:

'usestrict';

constassert=require('assert');

consttrueThing=true;

describe('Truethings',()=>{

it('aretrue',()=>{

assert.equal(trueThing,true);

TheBDDinterfacealsooffersthealiascontextfordescribeandspecifyforit.

Testfixturescanbehandledusingbeforeandafterforsuite-widefixturesandbeforeEachandafterEachforper-testfixtures:

describe('Falsethings',()=>{

letfalseThing;

before(()=>{

falseThing=!true;

it('arefalse',()=>{

assert.equal(falseThing,false);

TDDinterface

TheTDDinterfacedefinestestsuitesusingthesuitefunctionandeachtestcaseisdefinedusingthetestfunction:

'usestrict';

suite('Truethings',()=>{

test('aretrue',()=>{

Writingtests

TestfixturescanbehandledusingsuiteSetupandsuiteTeardownforsuite-widefixturesandsetupandteardownforper-testfixtures:

suite('Falsethings',()=>{

letfalseThing;

suiteSetup(()=>{

falseThing=!true;

test('arefalse',()=>{

Exportsinterface

TheExportsinterfacedefinestestcasesasmethodsofplainobjectpropertiesofthemodule.exportsobject:

'usestrict';

exports['Truethings']={

'aretrue':function(){

Thekeysbefore,after,beforeEachandafterEacharespecial-casedandbehavelikethecorrespondingfunctionsintheBDDinterface:

letfalseThing;

exports['Falsethings']={

before(){

falseThing=false;

'arefalse':function(){

Assertions

ArangoDBprovidestwobundledmodulestodefineassertions:

assertcorrespondstotheNode.jsassertmodule,providinglow-levelassertionsthatcanoptionallyspecifyanerrormessage.

chaiisthepopularChaiAssertionLibrary,providingbothBDDandTDDstyleassertionsusingafamiliarsyntax.

Writingtests

Cross-OriginResourceSharing(CORS)TouseCORSinyourFoxxservicesyoufirstneedtoconfigureArangoDBforCORS.Asof3.2Foxxwillthenautomaticallywhitelistallresponseheadersastheyareused.

IfyouwantmorecontroloverthewhitelistorareusinganolderversionofArangoDByoucansetthefollowingresponseheadersinyourrequesthandler:

access-control-expose-headers:acomma-separatedlistofresponseheaders.Thisdefaultstoalistofallheaderstheresponseisactuallyusing(butnotincludinganyaccess-controlheaders).

access-control-allow-credentials:canbesetto"false"toforbidexposingcookies.ThedefaultvaluedependsonwhetherArangoDBtruststheorigin.Seethenotesonhttp.trusted-origin.

NotethatitisnotpossibletooverridetheseheadersfortheCORSpreflightresponse.Itisthereforenotpossibletoacceptcredentialsorcookiesonlyforindividualroutes,servicesordatabases.TheoriginneedstobetrustedaccordingtothegeneralArangoDBconfiguration(seeabove).

CrossOrigin

FoxxscriptsandqueuedjobsFoxxletsyoudefinescriptsthatcanbeexecutedaspartoftheinstallationandremovalprocess,invokedmanuallyorscheduledtorunatalatertimeusingthejobqueue.

Toregisteryourscript,justaddascriptssectiontoyourservicemanifest:

"scripts":{

"setup":"scripts/setup.js",

"send-mail":"scripts/send-mail.js"

Thescriptsyoudefineinyourservicemanifestcanbeinvokedfromthewebinterfaceintheservice'ssettingspagewiththeScriptsdropdown.

Youcanalsousethescriptsasqueuedjobs:

'usestrict';

constqueues=require('@arangodb/foxx/queues');

queues.get('default').push(

{mount:'/my-service-mount-point',name:'send-mail'},

{to:'user@example.com',body:'Hello'}

ScriptargumentsandreturnvaluesIfthescriptwasinvokedwithanyarguments,youcanaccessthemusingthemodule.context.argvarray.

Toreturndatafromyourscript,youcanassignthedatatomodule.exportsasusual.PleasenotethatthisdatawillbeconvertedtoJSON.

Anyerrorsraisedbythescriptwillbehandleddependingonhowthescriptwasinvoked:

ifthescriptwasinvokedfromtheHTTPAPI(e.g.usingthewebinterface),itwillreturnanerrorresponseusingtheexception'sstatusCodepropertyifspecifiedor500.ifthescriptwasinvokedfromaFoxxjobqueue,thejob'sfailurecounterwillbeincrementedandthejobwillberescheduledormarkedasfailedifnoattemptsremain.

Examples

Let'ssayyouwanttodefineascriptthattakestwonumericvaluesandreturnstheresultofmultiplyingthem:

'usestrict';

constargv=module.context.argv;

assert.equal(argv.length,2,'Expectedexactlytwoarguments');

assert.equal(typeofargv[0],'number','Expectedfirstargumenttobeanumber');

assert.equal(typeofargv[1],'number','Expectedsecondargumenttobeanumber');

module.exports=argv[0]*argv[1];

LifecycleScriptsFoxxrecognizeslifecyclescriptsiftheyaredefinedandwillinvokethemduringtheinstallation,updateandremovalprocessoftheserviceifyouwant.

Thefollowingscriptsarecurrentlyrecognizedaslifecyclescriptsbytheirname:"setup"and"teardown".

SetupScript

ThesetupscriptwillbeexecutedwithoutargumentsduringtheinstallationofyourFoxxservice.

Thesetupscriptmaybeexecutedmorethanonceandshouldthereforebetreatedasreentrant.Runningthesamesetupscriptagainshouldnotresultinanyerrorsorduplicatedata.

Thesetupscriptistypicallyusedtocreatecollectionsyourserviceneedsorinsertseeddatalikeinitialadministrativeuseraccountsandsoon.

Examples

'usestrict';

consttextsCollectionName=module.context.collectionName('texts');

//`textsCollectionName`isnowtheprefixednameofthisservice's"texts"collection.

//e.g."example_texts"iftheservicehasbeenmountedat`/example`

if(db._collection(textsCollectionName)===null){

constcollection=db._create(textsCollectionName);

collection.save({text:'entry1fromcollectiontexts'});

}else{

console.debug(`collection${texts}alreadyexists.Leavingituntouched.`);

TeardownScript

TheteardownscriptwillbeexecutedwithoutargumentsduringtheremovalofyourFoxxservice.

Itcanalsooptionallybeexecutedbeforeupgradinganservice.

Thisscripttypicallyremovesthecollectionsand/ordocumentscreatedbyyourservice'ssetupscript.

Examples

'usestrict';

consttextsCollection=module.context.collection('texts');

if(textsCollection){

textsCollection.drop();

Queuesconstqueues=require('@arangodb/foxx/queues')

Foxxallowsdefiningjobqueuesthatletyouperformsloworexpensiveactionsasynchronously.Thesequeuescanbeusedtosende-mails,callexternalAPIsorperformotheractionsthatyoudonotwanttoperformdirectlyorwanttoretryonfailure.

enableordisabletheFoxxqueuesfeature--foxx.queuesflagIftrue,theFoxxqueueswillbeavailableandjobsinthequeueswillbeexecutedasynchronously.Thedefaultistrue.Whensettofalsethequeuemanagerwillbedisabledandanyjobsarepreventedfrombeingprocessed,whichmayreduceCPUloadabit.PleasenotethatFoxxjobqueuesaredatabase-specific.Queuesandjobsarealwaysrelativetothedatabaseinwhichtheyarecreatedoraccessed.

pollintervalforFoxxqueues--foxx.queues-poll-intervalvalueThepollintervalfortheFoxxqueuesmanager.Thevalueisspecifiedinseconds.LowervalueswillmeanmoreimmediateandmorefrequentFoxxqueuejobexecution,butwillmakethequeuethreadwakeupandquerythequeuesmoreoften.Whensettoalowvalue,thequeuethreadmightcauseCPUload.Thedefaultis1second.IfFoxx

queuesarenotusedmuch,thenthisvaluemaybeincreasedtomakethequeuesthreadwakeupless.Forthelow-levelfunctionalityseethechapteronthetaskmanagementmodule.

Creatingorupdatingaqueue

queues.create(name,[maxWorkers]):Queue

Returnsthequeueforthegivenname.Ifthequeuedoesnotexist,anewqueuewiththegivennamewillbecreated.IfaqueuewiththegivennamealreadyexistsandmaxWorkersisset,thequeue'smaximumnumberofworkerswillbeupdated.Thequeuewillbecreatedinthecurrentdatabase.

Arguments

name:string

Nameofthequeuetocreate.

maxWorkers:number(Default:1)

Themaximumnumberofworkers.

Examples

//Createaqueuewiththedefaultnumberofworkers(i.e.one)

constqueue1=queues.create("my-queue");

//Createaqueuewithagivennumberofworkers

constqueue2=queues.create("another-queue",2);

//Updatethenumberofworkersofanexistingqueue

constqueue3=queues.create("my-queue",10);

//queue1andqueue3refertothesamequeue

assertEqual(queue1,queue3);

Fetchinganexistingqueuequeues.get(name):Queue

Returnsthequeueforthegivenname.Ifthequeuedoesnotexistanexceptionisthrowninstead.

Thequeuewillbelookedupinthecurrentdatabase.

Arguments

name:string

Nameofthequeuetofetch.

Examples

Ifthequeuedoesnotyetexistanexceptionisthrown:

queues.get("some-queue");

//Error:Queuedoesnotexist:some-queue

//at...

Otherwisethequeuewillbereturned:

constqueue1=queues.create("some-queue");

constqueue2=queues.get("some-queue");

assertEqual(queue1,queue2);

Deletingaqueue

queues.delete(name):boolean

Returnstrueifthequeuewasdeletedsuccessfully.Ifthequeuedidnotexist,itreturnsfalseinstead.Thequeuewillbelookedupanddeletedinthecurrentdatabase.

Whenaqueueisdeleted,jobsonthatqueuewillnolongerbeexecuted.

Deletingaqueuewillnotdeleteanyjobsonthatqueue.

Arguments

name:string

Nameofthequeuetodelete.

Examples

constqueue=queues.create("my-queue");

queues.delete("my-queue");//true

queues.delete("my-queue");//false

Addingajobtoaqueue

queue.push(script,data,[opts]):string

Thejobwillbeaddedtothespecifiedqueueinthecurrentdatabase.

Returnsthejobid.

Arguments

script:object

Ajobtypedefinition,consistingofanobjectwiththefollowingproperties:

name:string

Nameofthescriptthatwillbeinvoked.

mount:string

Mountpathoftheservicethatdefinesthescript.

backOff:Function|number(Default:1000)

Eitherafunctionthattakesthenumberoftimesthejobhasfailedbeforeasinputandreturnsthenumberofmillisecondstowaitbeforetryingthejobagain,orthedelaytobeusedtocalculateanexponentialback-off,or0fornodelay.

maxFailures:number|Infinity(Default:0):

Numberoftimesasinglerunofajobwillbere-triedbeforeitismarkedas"failed".AnegativevalueorInfinitymeansthatthejobwillbere-triedonfailureindefinitely.

Schematovalidateajob'sdataagainstbeforeenqueuingthejob.

preprocess:Function(optional)

Functiontopre-processajob's(validated)databeforeserializingitinthequeue.

data:any

Jobdataofthejob;mustbeserializabletoJSON.

opts:object(optional)

Objectwithanyofthefollowingproperties:

success:Function(optional)

Functiontobecalledafterthejobhasbeencompletedsuccessfully.

failure:Function(optional)

Functiontobecalledafterthejobhasfailedtoomanytimes.

delayUntil:number|Date(Default:Date.now())

Timestampinmilliseconds(orDateinstance)untilwhichtheexecutionofthejobshouldbedelayed.

backOff:Function|number(Default:1000)

Seescript.backOff.

maxFailures:number|Infinity(Default:0):

Seescript.maxFailures.

repeatTimes:number|Function(Default:0)

Ifsettoapositivenumber,thejobwillberepeatedthismanytimes(notcountingrecoverywhenusingmaxFailures).IfsettoanegativenumberorInfinity,thejobwillberepeatedindefinitely.Ifsetto0thejobwillnotberepeated.

repeatUntil:number|Date(optional)

Ifthejobissettoautomaticallyrepeat,thiscanbesettoatimestampinmilliseconds(orDateinstance)afterwhichthejobwillnolongerrepeat.Settingthisvaluetozero,anegativevalueorInfinityhasnoeffect.

repeatDelay:number(Default:0)

Ifthejobissettoautomaticallyrepeat,thiscanbesettoanon-negativevaluetosetthenumberofmillisecondsforwhichthejobwillbedelayedbeforeitisstartedagain.

NotethatifyoupassafunctionforthebackOffcalculation,successcallbackorfailurecallbackoptionsthefunctionwillbeserializedtothedatabaseasastringandthereforemustnotrelyonanyexternalscopeorexternalvariables.

Whenthejobissettoautomaticallyrepeat,thefailurecallbackwillonlybeexecutedwhenarunofthejobhasfailedmorethanmaxFailurestimes.NotethatifthejobfailsandmaxFailuresisset,itwillberescheduledaccordingtothebackOffuntilithaseitherfailedtoomanytimesorcompletedsuccessfullybeforebeingscheduledaccordingtotherepeatDelayagain.RecoveryattemptsbymaxFailuresdonotcounttowardsrepeatTimes.

Thesuccessandfailurecallbacksreceivethefollowingarguments:

result:any

Thereturnvalueofthescriptforthecurrentrunofthejob.

jobData:any

Thedatapassedtothismethod.

job:object

ArangoDBdocumentrepresentingthejob'scurrentstate.

Examples

Let'ssaywehaveanservicemountedat/mailerthatprovidesascriptcalledsend-mail:

'usestrict';

constqueue=queues.create('my-queue');

queue.push(

{mount:'/mailer',name:'send-mail'},

{to:'hello@example.com',body:'Helloworld'}

Thiswillnotwork,becauselogwasdefinedoutsidethecallbackfunction(thecallbackmustbeserializabletoastring):

//WARNING:THISDOESNOTWORK!

'usestrict';

constlog=require('console').log;//outsidethecallback'sfunctionscope

queue.push(

{to:'hello@example.com',body:'Helloworld'},

{success:function(){

log('Yay!');//throws'logisnotdefined'

Here'sanexampleofajobthatwillbeexecutedevery5secondsuntiltomorrow:

'usestrict';

constqueues=require('@arangodb/foxx').queues;

queue.push(

{to:'hello@example.com',body:'Helloworld'},

repeatTimes:Infinity,

repeatUntil:Date.now()+(24*60*60*1000),

repeatDelay:5*1000

Fetchingajobfromthequeue

queue.get(jobId):Job

Createsaproxyobjectrepresentingajobwiththegivenjobid.

Thejobwillbelookedupinthespecifiedqueueinthecurrentdatabase.

ReturnsthejobforthegivenjobId.Propertiesofthejobobjectwillbefetchedwhenevertheyarereferencedandcannotbemodified.

Arguments

jobId:string

Theidofthejobtocreateaproxyobjectfor.

Examples

constjobId=queue.push({mount:'/logger',name:'log'},'HelloWorld!');

constjob=queue.get(jobId);

assertEqual(job.id,jobId);

Deletingajobfromthequeue

queue.delete(jobId):boolean

Deletesajobwiththegivenjobid.Thejobwillbelookedupanddeletedinthespecifiedqueueinthecurrentdatabase.

Arguments

jobId:string

Theidofthejobtodelete.

Returnstrueifthejobwasdeletedsuccessfully.Ifthejobdidnotexistitreturnsfalseinstead.

Fetchinganarrayofjobsinaqueue

Examples

constlogScript={mount:'/logger',name:'log'};

queue.push(logScript,'HelloWorld!',{delayUntil:Date.now()+50});

assertEqual(queue.pending(logScript).length,1);

//50mslater...

assertEqual(queue.pending(logScript).length,0);

assertEqual(queue.progress(logScript).length,1);

//evenlater...

assertEqual(queue.progress(logScript).length,0);

assertEqual(queue.complete(logScript).length,1);

Fetchinganarrayofpendingjobsinaqueuequeue.pending([script]):Array<string>

Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus"pending",optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.

Arguments

script:object(optional)

name:stringNameofthescript.

mount:stringMountpathoftheservicedefiningthescript.

Fetchinganarrayofjobsthatarecurrentlyinprogress

queue.progress([script])

Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus"progress",optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.

Arguments

Fetchinganarrayofcompletedjobsinaqueuequeue.complete([script]):Array<string>

Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus"complete",optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.

Arguments

Fetchinganarrayoffailedjobsinaqueue

queue.failed([script]):Array<string>

Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus"failed",optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.

Arguments

Fetchinganarrayofalljobsinaqueue

queue.all([script]):Array<string>

Returnsanarrayofjobidsofalljobsinthegivenqueue,optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.

Arguments

Abortingajobjob.abort():void

Abortsanon-completedjob.

Setsajob'sstatusto"failed"ifitisnotalready"complete",withoutcallingthejob'sonFailurecallback.

Migrating2.xservicesto3.0WhenmigratingservicesfromolderversionsofArangoDBitisgenerallyrecommendedyoumakesuretheyworkinlegacycompatibilitymode,whichcanalsoserveasastop-gapsolution.

ThischapteroutlinesthemajordifferencesintheFoxxAPIbetweenArangoDB2.8andArangoDB3.0.

Generalchanges

TheconsoleobjectinlaterversionsofArangoDB2.ximplementedaspecialFoxxconsoleAPIandwouldoptionallylogmessagestoacollection.ArangoDB3.0restorestheoriginalbehaviourwhereconsoleisthesameobjectavailablefromtheconsolemodule.

Migrating2.xservices

Migratingfrompre-2.8WhenmigratingfromaversionolderthanArangoDB2.8pleasenotethatstartingwithArangoDB2.8thebehaviouroftherequirefunctionmorecloselymimicsthebehaviourobservedinNode.jsandmodulebundlersforbrowsers,e.g.:

Inafile/routes/examples.js(relativetotherootfolderoftheservice):

require('./my-module')willbeattemptedtoberesolvedinthefollowingorder:

1. /routes/my-module(relativetoserviceroot)2. /routes/my-module.js(relativetoserviceroot)3. /routes/my-module.json(relativetoserviceroot)4. /routes/my-module/index.js(relativetoserviceroot)5. /routes/my-module/index.json(relativetoserviceroot)

require('lodash')willbeattemptedtoberesolvedinthefollowingorder:

1. /routes/node_modules/lodash(relativetoserviceroot)2. /node_modules/lodash(relativetoserviceroot)3. ArangoDBmodulelodash4. Nodecompatibilitymodulelodash5. BundledNPMmodulelodash

require('/abs/path')willbeattemptedtoberesolvedinthefollowingorder:

1. /abs/path(relativetofilesystemroot)2. /abs/path.js(relativetofilesystemroot)3. /abs/path.json(relativetofilesystemroot)4. /abs/path/index.js(relativetofilesystemroot)5. /abs/path/index.json(relativetofilesystemroot)

ThisbehaviourisincompatiblewiththesourcecodegeneratedbytheFoxxgeneratorinthewebinterfacebeforeArangoDB2.8.

Note:Theorg/arangodbmoduleisaliasedtothenewname@arangodbinArangoDB3.0.0andthe@arangodbmodulewasaliasedtotheoldnameorg/arangodbinArangoDB2.8.0.Eitheronewillworkin2.8and3.0butoutsideoflegacyservicesyoushoulduse@arangodbgoingforward.

FoxxqueueInArangoDB2.6FoxxintroducedanewwaytodefinequeuedjobsusingFoxxscriptstoreplacethefunction-basedjobtypedefinitionswhichwerecausingproblemswhenrestartingtheserver.Thefunction-basedjobshavebeenremovedin2.7andarenolongersupportedatall.

CoffeeScriptArangoDB3.0nolongerprovidesbuilt-insupportforCoffeeScriptsourcefiles,eveninlegacycompatibilitymode.IfyouwanttouseanalternativelanguagelikeCoffeeScript,makesuretopre-compiletherawsourcefilestoJavaScriptandusethecompiledJavaScriptfilesintheservice.

TherequestmoduleThe@arangodb/requestmodulewhenusedwiththejsonoptionpreviouslyoverwrotethestringinthebodypropertyoftheresponseobjectoftheresponsewiththeparsedJSONbody.In2.8thiswaschangedsotheparsedJSONbodyisaddedasthejsonpropertyoftheresponseobjectinadditiontooverwritingthebodyproperty.In3.0andlater(includinglegacycompatibilitymode)thebodypropertyisnolongeroverwrittenandmustusethejsonpropertyinstead.Notethatthisonlyaffectscodeusingthejsonoptionwhenmakingtherequest.

BundledNPMmodules

ThebundledNPMmoduleshavebeenupgradedandmayincludebackwards-incompatiblechanges,especiallytheAPIofjoihaschangedseveraltimes.Ifindoubtyoushouldbundleyourownversionsofthesemodulestoensurespecificversionswillbeused.

Theutilitymodulelodashisnowavailableandshouldbeusedinsteadofunderscore,butbothmoduleswillcontinuetobeprovided.

ManifestManyofthefieldsthatwererequiredinArangoDB2.xarenowoptionalandcanbesafelyomitted.

ToavoidcompatibilityproblemswithfutureversionsofArangoDByoushouldalwaysspecifytheenginesfield,e.g.:

"engines":{

"arangodb":"^3.0.0"

Controllers&exports

PreviouslyFoxxdistinguishedbetweenexportsandcontrollers,eachofwhichcouldbespecifiedasanobject.InArangoDB3.0thesehavebeenmergedintoasinglemainfieldspecifyinganentryfile.

Theeasiestwaytomigrateservicesusingmultipleexportsand/orcontrollersistocreateaseparateentryfilethatimportsthesefiles:

Old(manifest.json):

"exports":{

"doodads":"doodads.js",

"dingbats":"dingbats.js"

"controllers":{

"/doodads":"routes/doodads.js",

"/dingbats":"routes/dingbats.js",

"/":"routes/root.js"

New(manifest.json):

"main":"index.js"

New(index.js):

'usestrict';

module.context.use('/doodads',require('./routes/doodads'));

module.context.use('/dingbats',require('./routes/dingbats'));

module.context.use('/',require('./routes/root'));

module.exports={

doodads:require('./doodads'),

dingbats:require('./dingbats')

IndexredirectIfyoupreviouslydidnotdefinethedefaultDocumentfield,pleasenotethatinArangoDB3.0thefieldwillnolongerdefaulttothevalueindex.htmlwhenomitted:

//nodefaultDocument

manifest.json

"defaultDocument":"index.html"

ThisalsomeansitisnolongernecessarytospecifythedefaultDocumentfieldwithanemptyvaluetopreventtheredirectandbeabletoserverequestsatthe/(root)pathofthemountpoint:

"defaultDocument":""

//nodefaultDocument

AssetsTheassetsfieldisnolongersupportedinArangoDB3.0outsideoflegacycompatibilitymode.

Ifyoupreviouslyusedthefieldtoserveindividualfilesas-isyoucansimplyusethefilesfieldinstead:

"assets":{

"client.js":{

"files":["assets/client.js"],

"contentType":"application/javascript"

"files":{

"client.js":{

"path":"assets/client.js",

"type":"application/javascript"

Ifyoureliedonbeingabletospecifymultiplefilesthatshouldbeconcatenated,youwillhavetousebuildtoolsoutsideofArangoDBtopreparethesefilesaccordingly.

RootelementTherootElementfieldisnolongersupportedandhasbeenremovedentirely.

IfyourcontrollersreliedonthisfieldbeingavailableyouneedtoadjustyourschemasandroutestobeabletohandlethefullJSONstructureofincomingdocuments.

Systemservices

manifest.json

TheisSystemfieldisnolongersupported.ThepresenceorabsenceofthefieldhadnoeffectinmostrecentversionsofArangoDB2.xandhasnowbeenremovedentirely.

manifest.json

TheapplicationcontextTheglobalapplicationContextvariableavailableinFoxxmoduleshasbeenreplacedwiththecontextattributeofthemodulevariable.Forconsistencyitisnowreferredtoastheservicecontextthroughoutthisdocumentation.

SomemethodsoftheservicecontexthavechangedinArangoDB3.0:

fileName()nowbehaveslikepath()didinArangoDB2.xpath()hasbeenremoved(usefileName()instead)foxxFileName()hasbeenremoved(usefileName()instead)

Additionallytheversionandnameattributeshavebeenremovedandcannowonlybeaccessedviathemanifestattribute(asmanifest.versionandmanifest.name).Notethatthecorrespondingmanifestfieldsarenowoptionalandmaybeomitted.

Theoptionsattributehasalsobeenremovedasitshouldbeconsideredanimplementationdetail.Youshouldinsteadaccessthedependenciesandconfigurationattributesdirectly.

Theinternal_prefixattribute(whichwasanaliasforbasePath)andtheinternalcommentandclearCommentsmethods(whichwereusedbythemagicaldocumentationcommentsinArangoDB2.x)havealsobeenremoved.

Theinternal_serviceattribute(whichprovidesaccesstotheserviceitself)hasbeenrenamedtoservice.

applicationContext

RepositoriesandmodelsPreviouslyFoxxwasheavilybuiltaroundtheconceptofrepositoriesandmodels,whichprovidedcomplexbutrarelynecessaryabstractionsontopofArangoDBcollectionsanddocuments.InArangoDB3.0thesehavebeenremovedentirely.

Repositoriesvscollections

RepositoriesmostlywrappedmethodsthatalreadyexistedonArangoDBcollectionobjectsandprimarilydealtwithconvertingbetweenplainArangoDBdocumentsandFoxxmodelinstances.InArangoDB3.0youcansimplyusethesecollectionsdirectlyandtreatdocumentsasplainJavaScriptobjects.

'usestrict';

constFoxx=require('org/arangodb/foxx');

constmyRepo=newFoxx.Repository(

applicationContext.collection('myCollection'),

{model:Foxx.Model}

constmodels=myRepo.byExample({color:'green'});

res.json(models.map(function(model){

returnmodel.forClient();

'usestrict';

constmyDocs=module.context.collection('myCollection');

constdocs=myDocs.byExample({color:'green'});

res.json(docs);

Schemavalidation

ThemainpurposeofmodelsinArangoDB2.xwastovalidateincomingdatausingjoischemas.InmorerecentversionsofArangoDB2.xitwasalreadypossibletopasstheseschemasdirectlyinmostplaceswhereamodelwasexpectedasanargument.Theonlydifferenceisthatschemasshouldnowbeconsideredthedefault.

IfyoupreviouslyreliedontheautomaticvalidationofFoxxmodelinstanceswhensettingattributesorinstantiatingmodelsfromuntrusteddata,youcansimplyusetheschema'svalidatemethoddirectly.

'usestrict';

constmySchema={

size:joi.number().required()

constMyModel=Foxx.Model.extend({schema:mySchema});

constmodel=newMyModel(req.json());

if(!model.isValid){

res.status(400);

res.write('Badrequest');

return;

'usestrict';

//Notethisisnowwrappedinajoi.object()

constmySchema=joi.object({

}).required();

constresult=mySchema.validate(req.body);

if(result.errors){

res.status(400);

res.write('Badrequest');

return;

Migratingmodels

Whilemostusecasesformodelscannowbereplacedwithplainjoischemas,thereisstilltheconceptofa"model"inFoxxinArangoDB3.0althoughitisquitedifferentfromFoxxmodelsinArangoDB2.x.

AmodelinFoxxnowreferstoaplainJavaScriptobjectwithanoptionalschemaattributeandtheoptionalmethodsforClientandfromClient.Modelscanbeusedinsteadofplainjoischemastodefinerequestandresponsebodiesbuttherearenomodel"instances"inArangoDB3.0.

'usestrict';

const_=require('underscore');

constMyModel=Foxx.Model.extend({

schema:{

forClient(){

return_.omit(this.attributes,['_key','_id','_rev']);

ctrl.get(/*...*/)

.bodyParam('body',{type:MyModel});

'usestrict';

const_=require('lodash');

constMyModel={

schema:joi.object({

}).required(),

forClient(data){

return_.omit(data,['_key','_id','_rev']);

router.get(/*...*/)

.body(MyModel);

TriggersWhensaving,updating,replacingordeletingmodelsinArangoDB2.xusingtherepositorymethodstherepositoryandmodelwouldfireeventsthatcouldbesubscribedtoinordertoperformside-effects.

Notethatevenin2.xtheseeventswouldnotfirewhenusingqueriesormanipulatingdocumentsinanyotherwaythanusingthespecificrepositorymethodsthatoperatedonindividualdocuments.

ThisbehaviourisnolongeravailableinArangoDB3.0butcanbeemulatedbyusinganEventEmitterdirectlyifitisnotpossibletosolvetheproblemdifferently:

'usestrict';

constMyModel=Foxx.Model.extend({

afterRemove(){

console.log(this.get('name'),'wasremoved');

constmodel=myRepo.firstExample({name:'myName'});

myRepo.remove(model);

//->"myNamewasremovedsuccessfully"

'usestrict';

constEventEmitter=require('events');

constemitter=newEventEmitter();

emitter.on('afterRemove',function(doc){

console.log(doc.name,'wasremoved');

constdoc=myDocs.firstExample({name:'myName'});

myDocs.remove(doc);

emitter.emit('afterRemove',doc);

Orsimply:

'usestrict';

functionafterRemove(doc){

console.log(doc.name,'wasremoved');

constdoc=myDocs.firstExample({name:'myName'});

myDocs.remove(doc);

afterRemove(doc);

ControllersvsroutersFoxxControllershavebeenreplacedwithrouters.Thisismorethanacosmeticchangeastherearesignificantdifferencesinbehaviour:

Controllerswereautomaticallymountedwhenthefiledefiningthemwasexecuted.Routersneedtobeexplicitlymountedusingthemodule.context.usemethod.Routerscanalsobeexported,importedandevennested.Thismakesiteasiertosplitupcomplexroutingtreesacrossmultiplefiles.

'usestrict';

constctrl=newFoxx.Controller(applicationContext);

ctrl.get('/hello',function(req,res){

'usestrict';

constcreateRouter=require('org/arangodb/foxx/router');

//Ifyouareimportingthisfilefromyourentryfile("main"):

module.exports=router;

//Otherwise:module.context.use(router);

router.get('/hello',function(req,res){

Somegeneralchangesinbehaviourthatmighttripyouup:

WhenspecifyingpathparameterswithschemasFoxxwillnowignoretherouteiftheschemadoesnotmatch(i.e./hello/foxxwillnolongermatch/hello/:numifnumspecifiesaschemathatdoesn'tmatchthevalue"foxx").Withcontrollersthiscouldpreviouslyresultinusersseeinga400(badrequest)errorwhentheyshouldinsteadbeserveda404(notfound)response.

WhenarequestismadewithanHTTPverbnotsupportedbyanendpoint,Foxxwillnowrespondwitha405(methodnotallowed)errorwithanappropriateAllowedheaderlistingthesupportedHTTPverbsforthatendpoint.

FoxxwillnolongerparseyourJSDoccommentstogenerateroutedocumentation(usethesummaryanddescriptionmethodsoftheendpointinstead).

TheapiDocumentationmethodnowlivesontheservicecontextandbehavesslightlydifferently.

ThereisnorouterequivalentfortheactivateAuthenticationandactivateSessionsmethods.Insteadyoushouldusethesessionmiddleware(seethesectiononsessionsbelow).

Thereisnodelaliasforthedeletemethodonrouters.IthasalwaysbeensafetousekeywordsasmethodnamesinFoxx,sotheuseofthisaliaswasalreadydiscouragedbefore.

TheallRoutesproxyisnolotavailableonroutersbutcaneasilybereplacedwithmiddlewareorchildrouters.

Controllers

TherequestcontextWhendefiningarouteonacontrollerthecontrollerwouldreturnanobjectcalledrequestcontext.Routersreturnasimilarobjectcalledendpoint.Routersalsoreturnendpointswhenmountingchildroutersormiddleware,asdoestheusemethodoftheservicecontext.

ThemaindifferencesbetweenthenewendpointsandtheobjectsreturnedbycontrollersinpreviousversionsofArangoDBare:

bodyParamisnowsimplycalledbody;itisnolongerneccessaryorpossibletogivethebodyanameandtherequestbodywillnotshowupintherequestparameters.It'salsopossibletospecifyaMIMEtype

body,queryParamandpathParamnowtakepositionargumentsinsteadofanobject.Forspecificsseetheendpointdocumentation.

notesisnowcalleddescriptionandtakesasinglestringargument.

onlyIfandonlyIfAuthenticatedarenolongeravailable;theycanbeemulatedwithmiddlewareifnecessary:

ctrl.get(/*...*/)

.onlyIf(function(req){

if(!req.user){

thrownewError('Notauthenticated!');

router.use(function(req,res,next){

res.throw(403,'Notauthenticated!');

next();

router.get(/*...*/);

Controllers

ErrorhandlingTheerrorResponsemethodprovidedbycontrollerrequestcontextshasnoequivalentinrouterendpoints.Ifyouwanttohandlespecificerrortypeswithspecificstatuscodesyouneedtocatchthemexplicitly,eitherintherouteorinamiddleware:

ctrl.get('/puppies',function(req,res){

//Exceptionisthrownhere

.errorResponse(TooManyPuppiesError,400,'Somethingwentwrong!');

ctrl.get('/puppies',function(req,res){

//Exceptionisthrownhere

}catch(e){

if(!(einstanceofTooManyPuppiesError)){

throwe;

res.throw(400,'Somethingwentwrong!');

//The"error"methodmerelydocumentsthemeaning

//ofthestatuscodeandhasnoothereffect.

.error(400,'Throwniftherearetoomanypuppies.');

Notethaterrorscreatedwithhttp-errorsarestillhandledbyFoxxintelligently.Infactres.throwisjustahelpermethodforcreatingandthrowingtheseerrors.

Controllers

Before,afterandaroundThebefore,afterandaroundmethodscaneasilybereplacedbymiddleware:

letstart;

ctrl.before(function(req,res){

start=Date.now();

ctrl.after(function(req,res){

console.log('Requesthandledin',(Date.now()-start),'ms');

letstart=Date.now();

next();

console.log('Requesthandledin',(Date.now()-start),'ms');

Notethatunlikearoundmiddlewarereceivesthenextfunctionasthethirdargument(the"opts"argumenthasnoequivalent).

Controllers

RequestobjectsThenamesofsomeattributesoftherequestobjecthavebeenadjustedtomorecloselyalignwiththoseofthecorrespondingmethodsontheendpointobjectsandestablishedconventionsinotherJavaScriptframeworks:

req.urlParametersisnowcalledreq.pathParams

req.parametersisnowcalledreq.queryParams

req.params()isnowcalledreq.param()

req.requestTypeisnowcalledreq.method

req.compatibilityisnowcalledreq.arangoVersion

req.userisnowcalledreq.arangoUser

Someattributeshavebeenremovedorchanged:

req.cookieshasbeenremovedentirely(usereq.cookie(name))

req.requestBodyhasbeenremovedentirely(seebelow)

req.suffixisnowastringratherthananarray

Additionallythereq.serverandreq.clientattributesarenolongeravailable.Theinformationisnowexposedinawaythatcan(optionally)transparentlyhandleproxyforwardingheaders:

req.hostnamedefaultstoreq.server.address

req.portdefaultstoreq.server.port

req.remoteAddressdefaultstoclient.address

req.remotePortdefaultstoclient.port

Finally,thereq.cookiemethodnowtakesthesignedoptionsdirectly.

constsid=req.cookie('sid',{

signed:{

secret:'keyboardcat',

algorithm:'sha256'

constsid=req.cookie('sid',{

algorithm:'sha256'

RequestbodiesThereq.bodyisnolongeramethodandnolongerautomaticallyparsesJSONrequestbodiesunlessarequestbodywasdefined.Thereq.rawBodynowcorrespondstothereq.rawBodyBufferofArangoDB2.xandisalsonolongeramethod.

ctrl.post('/',function(req,res){

constdata=req.body();

Controllers

router.post('/',function(req,res){

constdata=req.body;

.body(['json']);

Orsimply:

constdata=req.body;

.body(joi.object().optional());

Multipartrequests

Thereq.requestPartsmethodhasbeenremovedentirely.Ifyouneedtoacceptmultipartrequestbodies,youcansimplydefinetherequestbodyusingamultipartMIMEtypelikemultipart/form-data:

ctrl.post('/',function(req,res){

constparts=req.requestParts();

constparts=req.body;

.body(['multipart/form-data']);

Controllers

ResponseobjectsTheresponseobjecthasalotofnewmethodsinArangoDB3.0butotherwiseremainssimilartotheresponseobjectofpreviousversions:

Theres.sendmethodbehavesverydifferentlyfromhowthemethodwiththesamenamebehavedinArangoDB2.x:theconversionnowtakestheresponsebodydefinitionoftherouteintoaccount.Thereisanewmethodres.writethatimplementstheoldbehaviour.

Notethatconsecutivecallstores.writewillappendtotheresponsebodyratherthanreplacingitlikeres.send.

Theres.contentTypepropertyisalsonolongeravailable.IfyouwanttosettheMIMEtypeoftheresponsebodytoanexplicitvalueyoushouldsetthecontent-typeheaderinstead:

res.contentType='application/json';

res.body=JSON.stringify(results);

res.set('content-type','application/json');

res.body=JSON.stringify(results);

Orsimply:

//setsthecontenttypetoJSON

//ifithasnotalreadybeenset

res.json(results);

Theres.cookiemethodnowtakesthesignedoptionsaspartoftheregularoptionsobject.

res.cookie('sid','abcdef',{

ttl:60*60,

signed:{

algorithm:'sha256'

res.cookie('sid','abcdef',{

ttl:60*60,

algorithm:'sha256'

Controllers

DependencyinjectionThereisnoequivalentoftheaddInjectormethodavailableinArangoDB2.xcontrollers.Mostusecasescanbesolvedbysimplyusingplainvariablesbutifyouneedsomethingmoreflexibleyoucanalsousemiddleware:

ctrl.addInjector('magicNumber',function(){

returnMath.random();

ctrl.get('/',function(req,res,injected){

res.json(injected.magicNumber);

functionmagicMiddleware(name){

return{

register(){

letmagic;

returnfunction(req,res,next){

if(!magic){

magic=Math.random();

req[name]=magic;

next();

router.use(magicMiddleware('magicNumber'));

res.json(req.magicNumber);

Orsimply:

constmagicNumber=Math.random();

res.json(magicNumber);

Controllers

SessionsThectrl.activateSessionsmethodandtherelatedutil-sessions-localFoxxservicehavebeenreplacedwiththeFoxxsessionsmiddleware.Itisnolongerpossibletousethebuilt-insessionstoragebutyoucansimplypassinanydocumentcollectiondirectly.

constlocalSessions=applicationContext.dependencies.localSessions;

constsessionStorage=localSessions.sessionStorage;

ctrl.activateSessions({

sessionStorage:sessionStorage,

cookie:{secret:'keyboardcat'}

ctrl.destroySession('/logout',function(req,res){

res.json({message:'Goodbye!'});

constsessionMiddleware=require('@arangodb/foxx/sessions');

constcookieTransport=require('@arangodb/foxx/sessions/transports/cookie');

router.use(sessionMiddleware({

transport:cookieTransport('keyboardcat')

router.post('/logout',function(req,res){

req.sessionStorage.clear(req.session);

res.json({message:'Goodbye!'});

Sessions

AuthandOAuth2Theutil-simple-authandutil-oauth2FoxxserviceshavebeenreplacedwiththeFoxxauthandFoxxOAuth2modules.Itisnolongernecessarytoinstalltheseservicesasdependenciesinordertousethefunctionality.

'usestrict';

constauth=applicationContext.dependencies.simpleAuth;

constvalid=auth.verifyPassword(authData,password);

'usestrict';

constcreateAuth=require('@arangodb/foxx/auth');

constauth=createAuth();//Usedefaultconfiguration

constvalid=auth.verifyPassword(authData,password);

AuthandOAuth2

FoxxqueriesThecreateQuerymethodhasbeenremoved.ItcanbetriviallyreplacedwithplainJavaScriptfunctionsanddirectcallstothedb._querymethod:

'usestrict';

constquery=Foxx.createQuery({

query:'FORuIN_usersSORTu.userASCRETURNu[@propName]',

params:['propName'],

transform:function(results,uppercase){

return(

uppercase

?results[0].toUpperCase()

:results[0].toLowerCase()

query('user',true);

'usestrict';

functionquery(propName,uppercase){

constresults=db._query(aql`

FORuIN_users

SORTu.userASC

RETURNu[${propName}]

return(

uppercase

?results[0].toUpperCase()

:results[0].toLowerCase()

query('user',true);

FoxxQueries

Legacycompatibilitymodefor2.8servicesArangoDB3continuestosupportFoxxserviceswrittenforArangoDB2.8byrunningtheminaspeciallegacycompatibilitymodethatprovidesaccesstosomeofthemodulesandAPIsnolongerprovidedin3.0andbeyond.

Note:LegacycompatibilitymodeisstrictlyintendedasatemporarystopgapsolutionforsupportingexistingserviceswhileupgradingtoArangoDB3.0andshouldnotbeconsideredapermanentfeatureofArangoDBorFoxx.

Inordertomarkanexistingserviceasalegacyservice,justmakesurethefollowingattributeisdefinedintheservicemanifest:

"engines":{

"arangodb":"^2.8.0"

ThissemanticversionrangedenotesthattheserviceisknowntoworkwithArangoDB2.8.0andsupportsallnewerversionsofArangoDBuptobutnotincluding3.0.0(noranydevelopmentversionof3.0.0andgreater).

Anysimilarversionrangethedoesnotinclude3.0.0orgreaterwillhavethesameeffect(e.g. 2.5.0willalsotriggerthelegacycompatibilitymode,aswill1.2.3,but>=2.8.0willnotasitindicatescompatibilitywithallversionsgreaterorequal2.8.0,notjustthosewithinthe2.xversionrange).

FeaturessupportedinlegacycompatibilitymodeLegacycompatibilitymodesupportstheoldmanifestformat,specifically:

mainisignoredcontrollerswillbemountedasin2.8exportswillbeexecutedasin2.8

AdditionallytheisSystemattributewillbeignoredifpresentbutdoesnotresultinawarninginlegacycompatibilitymode.

TheFoxxconsoleisavailableastheconsolepseudo-globalvariable(shadowingtheglobalconsoleobject).

TheservicecontextisavailableastheapplicationContextpseudo-globalvariableinthecontrollers,exports,scriptsandtestsasin2.8.Thefollowingadditionalpropertiesareavailableontheservicecontextinlegacycompatibilitymode:

path()isanaliasfor3.xfileName()(usingpath.jointobuildfilepaths)fileName()behavesasin2.x(usingfs.safeJointobuildfilepaths)foxxFileName()isanaliasfor2.xfileNameversionexposestheservicemanifest'sversionattributenameexposestheservicemanifest'snameattributeoptionsexposestheservice'srawoptions

Thefollowingmethodsareremovedontheservicecontextinlegacycompatibilitymode:

use()--use@arangodb/foxx/controllerinsteadapiDocumentation()--usecontroller.apiDocumentation()insteadregisterType()--notsupportedinlegacycompatibilitymode

Thefollowingmodulesthathavebeenremovedorreplacedin3.0.0areavailableinlegacycompatibilitymode:

@arangodb/foxx/authentication

@arangodb/foxx/console

@arangodb/foxx/controller

@arangodb/foxx/model

@arangodb/foxx/query

@arangodb/foxx/repository

@arangodb/foxx/schema

@arangodb/foxx/sessions

@arangodb/foxx/template_middleware

The@arangodb/foxxmodulealsoprovidesthesameexportsasin2.8,namely:

Controllerfrom@arangodb/foxx/controllercreateQueryfrom@arangodb/foxx/queryModelfrom@arangodb/foxx/modelRepositoryfrom@arangodb/foxx/repositorytoJSONSchemafrom@arangodb/foxx/schemagetExportsandrequireAppfrom@arangodb/foxx/managerqueuesfrom@arangodb/foxx/queues

Anyfeaturenotsupportedin2.8willalsonotworkinlegacycompatibilitymode.WhenmigratingfromanolderversionofArangoDBitisagoodideatomigratetoArangoDB2.8firstforaneasierupgradepath.

AdditionallypleasenotethedifferenceslaidoutinthechapterMigratingfrompre-2.8inthemigrationguide.

UsermanagementFoxxdoesnotprovideanyusermanagementoutoftheboxbutitisveryeasytorollyourownsolution:

Thesessionmiddlewareprovidesmechanismsforaddingsessionlogictoyourservice,usinge.g.acollectionorJSONWebTokenstostorethesessionsbetweenrequests.

Theauthmoduleprovidesutilitiesforbasicpasswordverificationandhashing.

Thefollowingexampleservicedemonstrateshowusermanagementcanbeimplementedusingthesebasicbuildingblocks.

Settingupthecollections

Let'ssaywewanttostoresessionsandusersincollections.Wecanusethesetupscripttomakesurethesecollectionsarecreatedbeforetheserviceismounted.

Firstaddasetupscripttoyourmanifestifitisn'talreadydefined:

"scripts":{

"setup":"scripts/setup.js"

Thencreatethesetupscriptwiththefollowingcontent:

'usestrict';

constsessions=module.context.collectionName('sessions');

constusers=module.context.collectionName('users');

if(!db._collection(sessions)){

db._createDocumentCollection(sessions);

if(!db._collection(users)){

db._createDocumentCollection(users);

db._collection(users).ensureIndex({

type:'hash',

fields:['username'],

unique:true

Creatingtherouter

Thefollowingmainfiledemonstratesbasicusermanagement:

'usestrict';

constcreateAuth=require('@arangodb/foxx/auth');

constsessionsMiddleware=require('@arangodb/foxx/sessions');

constauth=createAuth();

constusers=module.context.collection('users');

transport:'cookie'

router.get('/whoami',function(req,res){

Usermanagement

constuser=users.document(req.session.uid);

res.send({username:user.username});

}catch(e){

res.send({username:null});

.description('Returnsthecurrentlyactiveusername.');

router.post('/login',function(req,res){

//Thismayreturnauserobjectornull

constuser=users.firstExample({

username:req.body.username

constvalid=auth.verify(

//Pretendtovalidateevenifnouserwasfound

user?user.authData:{},

req.body.password

if(!valid)res.throw('unauthorized');

//Logtheuserin

req.session.uid=user._key;

res.send({sucess:true});

.body(joi.object({

username:joi.string().required(),

password:joi.string().required()

}).required(),'Credentials')

.description('Logsaregistereduserin.');

router.post('/logout',function(req,res){

if(req.session.uid){

req.session.uid=null;

res.send({success:true});

.description('Logsthecurrentuserout.');

router.post('/signup',function(req,res){

constuser=req.body;

//Createanauthenticationhash

user.authData=auth.create(user.password);

deleteuser.password;

constmeta=users.save(user);

Object.assign(user,meta);

}catch(e){

//Failedtosavetheuser

//We'llassumetheUniqueConstrainthasbeenviolated

res.throw('badrequest','Usernamealreadytaken',e);

//Logtheuserin

res.send({success:true});

.body(joi.object({

username:joi.string().required(),

password:joi.string().required()

}).required(),'Credentials')

.description('Createsanewuserandlogsthemin.');

Usermanagement

RelatedmodulesThesearesomeofthemodulesoutsideofFoxxyouwillfindusefulwhenwritingFoxxservices.

AdditionallytherearemodulesprovidingsomelevelofcompatibilitywithNode.jsaswellasanumberofbundledNPMmodules(likelodashandjoi).FormoreinformationonthesemodulesseetheJavaScriptmodulesappendix.

The @arangodbmodulerequire('@arangodb')

ThismoduleprovidesaccesstovariousArangoDBinternalsaswellasthreeofthemostimportantexportsnecessarytoworkwiththedatabaseinFoxx:db,aqlanderrors.

YoucanfindafulldescriptionofthismoduleintheArangoDBmoduleappendix.

The @arangodb/requestmodulerequire('@arangodb/request')

ThismoduleprovidesafunctionformakingHTTPrequeststoexternalservices.Notethatwhilethisallowscommunicatingwiththird-partyservicesitmayaffectdatabaseperformancebyblockingFoxxrequestsasArangoDBwaitsfortheremoteservicetorespond.Ifyouroutinelymakerequeststoslowexternalservicesandarenotdirectlyinterestedintheresponseitisprobablyabetterideatodelegatetheactualrequest/responsecycletoagatewayservicerunningoutsideArangoDB.

Youcanfindafulldescriptionofthismoduleintherequestmoduleappendix.

The @arangodb/general-graphmodulerequire('@arangodb/general-graph')

ThismoduleprovidesaccesstoArangoDBgraphdefinitionsandvariouslow-levelgraphoperationsinJavaScript.FormorecomplexqueriesitisprobablybettertouseAQLbutthismodulecanbeusefulinyoursetupandteardownscriptstocreateanddestroygraphdefinitions.

Formoreinformationseethechapteronthegeneralgraphmodule.

Relatedmodules

AuthenticationconstcreateAuth=require('@arangodb/foxx/auth');

Authenticatorsallowimplementingbasicpasswordmechanismusingsimplebuilt-inhashingfunctions.

ForafullexampleofsessionswithauthenticationandregistrationseetheexampleinthechapteronUserManagement.

CreatinganauthenticatorcreateAuth([options]):Authenticator

Createsanauthenticator.

Arguments

method:string(Default:"sha256")

Thehashingalgorithmtousetocreatepasswordhashes.Theauthenticatorwillbeabletoverifypasswordsagainsthashesusinganysupportedhashingalgorithm.Thisonlyaffectsnewhashescreatedbytheauthenticator.

Supportedvalues:

"sha1"

"sha224"

"sha256"

"sha384"

"sha512"

saltLength:number(Default:16)

Lengthofthesaltsthatwillbegeneratedforpasswordhashes.

Returnsanauthenticator.

Creatingauthenticationdataobjectsauth.create(password):AuthData

Createsanauthenticationdataobjectforthegivenpasswordwiththefollowingproperties:

method:string

Themethodusedtogeneratethehash.

salt:string

Arandomsaltusedtogeneratethishash.

hash:string

Thehashstringitself.

Arguments

password:string

Apasswordtohash.

Returnstheauthenticationdataobject.

Authentication

Validatingpasswordsagainstauthenticationdataobjectsauth.verify([hash,[password]]):boolean

Verifiesthegivenpasswordagainstthegivenhashusingaconstanttimestringcomparison.

Arguments

hash:AuthData(optional)

Aauthenticationdataobjectgeneratedwiththecreatemethod.

password:string(optional)

Apasswordtoverifyagainstthehash.

Returnstrueifthehashmatchesthegivenpassword.Returnsfalseotherwise.

Authentication

OAuth1.0aconstcreateOAuth1Client=require('@arangodb/foxx/oauth1');

TheOAuth1moduleprovidesabstractionsoverOAuth1.0aproviderslikeTwitter,XINGandTumblr.

Examples

Thefollowingextendstheusermanagementexample:

constoauth1=createOAuth1Client({

//We'lluseTwitterforthisexample

requestTokenEndpoint:'https://api.twitter.com/oauth/request_token',

authEndpoint:'https://api.twitter.com/oauth/authorize',

accessTokenEndpoint:'https://api.twitter.com/oauth/access_token',

activeUserEndpoint:'https://api.twitter.com/1.1/account/verify_credentials.json',

clientId:'keyboardcat',

clientSecret:'keyboardcat'

module.context.use('/oauth1',router);

//Seetheusermanagementexampleforsettingupthe

//sessionsandusersobjectsusedinthisexample

router.use(sessions);

router.post('/auth',function(req,res){

consturl=req.reverse('oauth1_callback');

constoauth_callback=req.makeAbsolute(url);

constrequestToken=oauth1.fetchRequestToken(oauth_callback);

if(requestToken.oauth_callback_confirmed!=='true'){

res.throw(500,'CouldnotfetchOAuthrequesttoken');

//Setrequesttokencookieforfiveminutes

res.cookie('oauth1_request_token',requestToken.oauth_token,{ttl:60*5});

//Redirecttotheprovider'sauthorizationURL

res.redirect(303,oauth1.getAuthUrl(requestToken.oauth_token));

router.get('/auth',function(req,res){

//MakesureCSRFcookiematchestheURL

constexpectedToken=req.cookie('oauth1_request_token');

if(!expectedToken||req.queryParams.oauth_token!==expectedToken){

res.throw(400,'CSRFmismatch.');

constauthData=oauth1.exchangeRequestToken(

req.queryParams.oauth_token,

req.queryParams.oauth_verifier

consttwitterToken=authData.oauth_token;

consttwitterSecret=authData.oauth_token_secret;

//Fetchtheactiveuser'sprofileinfo

constprofile=oauth1.fetchActiveUser(twitterToken,twitterSecret);

consttwitterId=profile.screen_name;

//TrytofindanexistinguserwiththeuserID

//(thisrequirestheuserscollection)

letuser=users.firstExample({twitterId});

if(user){

//UpdatethetwitterTokenifithaschanged

user.twitterToken!==twitterToken||

user.twitterSecret!==twitterSecret

users.update(user,{twitterToken,twitterSecret});

}else{

//Createanewuserdocument

user={

username:`twitter:${twitterId}`,

twitterId,

twitterToken

OAuth1.0a

//Logtheuserin(thisrequiresthesessionmiddleware)

req.session.twitterToken=authData.twitterToken;

req.session.twitterSecret=authData.twitterSecret;

//Redirecttothedefaultroute

res.redirect(303,req.makeAbsolute('/'));

},'oauth1_callback')

.queryParam('oauth_token',joi.string().optional())

.queryParam('oauth_verifier',joi.string().optional());

CreatinganOAuth1.0aclientcreateOAuth1Client(options):OAuth1Client

CreatesanOAuth1.0aclient.

Arguments

options:Object

requestTokenEndpoint:string

Thefully-qualifiedURLoftheprovider'sTemporaryCredentialsRequestendpoint.ThisURLisusedtofetchtheunauthenticatedtemporarycredentialsthatwillbeusedtogeneratetheauthorizationredirectfortheuser.

authEndpoint:string

Thefully-qualifiedURLoftheprovider'sResourceOwnerAuthorizationendpoint.ThisistheURLtheuserwillberedirectedtoinordertoauthorizetheOAuthconsumer(i.e.yourservice).

accessTokenEndpoint:string

Thefully-qualifiedURLoftheprovider'sTokenRequestendpoint.ThisURLisusedtoexchangetheauthenticatedtemporarycredentialsreceivedfromtheauthorizationredirectfortheactualtokencredentialsthatcanbeusedtomakerequeststotheAPIserver.

activeUserEndpoint:string(optional)

Thefully-qualifiedURLoftheprovider'sendpointforfetchingdetailsaboutthecurrentuser.

clientId:string

Theapplication'sClientID(orConsumerKey)fortheprovider.

clientSecret:string

Theapplication'sClientSecret(orConsumerSecret)fortheprovider.

signatureMethod:string(Default:"HMAC-SHA1")

ThecryptographicmethodthatwillbeusedtosignOAuth1.0arequests.Only"HMAC-SHA1-"and"PLAINTEXT"aresupportedatthistime.

Notethatmanyprovidersmaynotimplement"PLAINTEXT"asitexposestheClientSecretandoauth_token_secretinsteadofgeneratingasignature.

ReturnsanOAuth1.0aclientforthegivenprovider.

SettingupOAuth1.0aforTwitter

IfyouwanttouseTwitterastheOAuth1.0aprovider,usethefollowingoptions:

OAuth1.0a

requestTokenEndpoint:https://api.twitter.com/oauth/request_tokenauthEndpoint:https://api.twitter.com/oauth/authorizeaccessTokenEndpoint:https://api.twitter.com/oauth/access_tokenactiveUserEndpoint:https://api.twitter.com/1.1/account/verify_credentials.json

YoualsoneedtoobtainaclientIDandclientsecretfromTwitter:

1. CreatearegularaccountatTwitteroruseanexistingaccountyouown.2. VisittheTwitterApplicationManagementdashboardandsigninwithyourTwitteraccount.3. ClickonCreateNewAppandfollowtheinstructionsprovided.TheCallbackURLshouldmatchyouroauth_callbacklater.You

maybepromptedtoaddamobilephonenumbertoyouraccountandverifyit.4. OpentheKeysandAccessTonestab,thennotedowntheConsumerKeyandConsumerSecret.5. SettheoptionclientIdtotheConsumerKeyandtheoptionclientSecrettotheConsumerSecret.

Notethatifyouonlyneedread-onlyaccesstopublicinformation,youcanalsousetheclientIdandclientSecretdirectlywithoutOAuth1.0a.

SeeTwitterRESTAPIReferenceDocumentation.

SettingupOAuth1.0aforXING

IfyouwanttouseXINGastheOAuth1.0aprovider,usethefollowingoptions:

requestTokenEndpoint:https://api.xing.com/v1/request_tokenauthEndpoint:https://api.xing.com/v1/authorizeaccessTokenEndpoint:https://api.xing.com/v1/access_tokenactiveUserEndpoint:https://api.xing.com/v1/users/me

YoualsoneedtoobtainaclientIDandclientsecretfromXING:

1. CreatearegularaccountatXINGoruseanexistingaccountyouown.2. VisittheXINGDeveloperpageandsigninwithyourXINGaccount.3. ClickonCreateappandnotedowntheConsumerkeyandConsumersecret.4. SettheoptionclientIdtotheConsumerkeyandtheoptionclientSecrettotheConsumersecret.

SeeXINGDeveloperDocumentation.

SettingupOAuth1.0aforTumblr

IfyouwanttouseTumblrastheOAuth1.0aprovider,usethefollowingoptions:

requestTokenEndpoint:https://www.tumblr.com/oauth/request_tokenauthEndpoint:https://www.tumblr.com/oauth/authorizeaccessTokenEndpoint:https://www.tumblr.com/oauth/access_tokenactiveUserEndpoint:https://api.tumblr.com/v2/user/info

YoualsoneedtoobtainaclientIDandclientsecretfromTumblr:

1. CreatearegularaccountatTumblroruseanexistingaccountyouown.2. VisittheTumblrApplicationsdashboard.3. ClickonRegisterapplication,thenfollowtheinstructionsprovided.TheDefaultcallbackURLshouldmatchyouroauth_callback

later.4. NotedowntheOAuthConsumerKeyandSecretKey.Thesecretmaybehiddenbydefault.5. SettheoptionclientIdtotheOAuthConsumerKeyandtheoptionclientSecrettotheSecretKey.

SeeTumblrAPIDocumentation.

Fetchanunauthenticatedrequesttokenoauth1.fetchRequestToken(oauth_callback,opts)

Fetchesanoauth_tokenthatcanbeusedtocreateanauthorizationURLthatredirectstothegivenoauth_callbackonconfirmation.

OAuth1.0a

PerformsaPOSTresponsetotherequestTokenEndpoint.

Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.

Arguments

oauth_callback:string

Thefully-qualifiedURLofyourapplication'sOAuth1.0acallback.

opts:Object(optional)

Anobjectwithadditionalqueryparameterstoincludeintherequest.

SeeRFC5849.

Returnstheparsedresponseobject.

GettheauthorizationURLoauth1.getAuthUrl(oauth_token,opts):string

GeneratestheauthorizationURLfortheauthorizationendpoint.

Arguments

oauth_token:string

Theoauth_tokenpreviouslyreturnedbyfetchRequestToken.

opts:(optional)

AnobjectwithadditionalqueryparameterstoaddtotheURL.

SeeRFC5849.

Returnsafully-qualifiedURLfortheauthorizationendpointoftheproviderbyappendingtheoauth_tokenandanyadditionalargumentsfromoptstotheauthEndpoint.

Examples

constrequestToken=oauth1.fetchRequestToken(oauth_callback);

if(requestToken.oauth_callback_confirmed!=='true'){

thrownewError('ProvidercouldnotconfirmOAuth1.0callback');

constauthUrl=oauth1.getAuthUrl(requestToken.oauth_token);

Exchangeanauthenticatedrequesttokenforanaccesstokenoauth1.exchangeRequestToken(oauth_token,oauth_verifier,opts)

TakesapairofauthenticatedtemporarycredentialspassedtothecallbackURLbytheproviderandexchangesitforanoauth_tokenandoauth_token_secretthancanbeusedtoperformauthenticatedrequeststotheOAuth1.0aprovider.

PerformsaPOSTresponsetotheaccessTokenEndpoint.

Arguments

oauth_token:string

Theoauth_tokenpassedtothecallbackURLbytheprovider.

oauth_verifier:string

Theoauth_verifierpassedtothecallbackURLbytheprovider.

OAuth1.0a

SeeRFC5849.

Fetchtheactiveuseroauth1.fetchActiveUser(oauth_token,oauth_token_secret,opts):Object

Fetchesdetailsoftheactiveuser.

PerformsaGETresponsetotheactiveUserEndpoint.

ReturnsnulliftheactiveUserEndpointisnotconfigured.

Arguments

oauth_token:string

AnOAuth1.0aaccesstokenasreturnedbyexchangeRequestToken.

oauth_token_secret:string

AnOAuth1.0aaccesstokensecretasreturnedbyexchangeRequestToken.

SeeRFC5849.

Examples

constauthData=oauth1.exchangeRequestToken(oauth_token,oauth_verifier);

constuserData=oauth1.fetchActiveUser(authData.oauth_token,authData.oauth_token_secret);

Createanauthenticatedrequestobjectoauth1.createSignedRequest(method,url,parameters,oauth_token,oauth_token_secret)

CreatesarequestobjectthatcanbeusedtoperformarequesttotheOAuth1.0aproviderwiththeprovidedtokencredentials.

Arguments

method:string

HTTPmethodtherequestwilluse,e.g."POST".

url:string

Thefully-qualifiedURLoftheprovidertherequestwillbeperformedagainst.

TheURLmayoptionallycontainanynumberofqueryparameters.

parameters:string|Object|null

Anadditionalobjectorquerystringcontainingqueryparametersorbodyparametersthatwillbepartofthesignedrequest.

oauth_token:string

AnOAuth1.0aaccesstokenasreturnedbyexchangeRequestToken.

oauth_token_secret:string

OAuth1.0a

AnOAuth1.0aaccesstokensecretasreturnedbyexchangeRequestToken.

Returnsanobjectwiththreeproperties:

url:ThenormalizedURLwithoutanyqueryparameters.

qs:Anormalizedquerystringcontainingallparametersandqueryparameters.

headers:Anobjectcontainingthefollowingproperties:

accept:Thestring"application/json".

authorization:AnOAuthauthorizationheadercontainingallOAuthparametersandtherequestsignature.

Examples

Fetchalistoftweetsmentioning@arangodb:

constrequest=require('@arangodb/request');

constreq=oauth1.createSignedRequest(

'GET',

'https://api.twitter.com/1.1/search/tweets.json',

{q:'@arangodb'},

authData.oauth_token,

authData.oauth_token_secret

constres=request(req);

console.log(res.json.statuses);

Signingamorecomplexrequest:

consturl='https://api.example.com/v1/timeline?visible=public';

constparams={hello:'world',longcat:'islong'};

constreq=oauth1.createSignedRequest(

'POST',

url,//URLincludesaqueryparameterthatwillbesigned

params,//Requestbodyneedstobesignedtoo

authData.oauth_token,

authData.oauth_token_secret

constres=request.post(url,{

form:params,

headers:{

accept:'application/x-www-form-urlencoded',

//Authorizationheaderincludesthesignature

authorization:req.headers.authorization

console.log(res.json);

OAuth1.0a

OAuth2.0constcreateOAuth2Client=require('@arangodb/foxx/oauth2');

TheOAuth2moduleprovidesabstractionsoverOAuth2.0providerslikeFacebook,GitHubandGoogle.

Examples

Thefollowingextendstheusermanagementexample:

constcrypto=require('@arangodb/crypto');

constoauth2=createOAuth2Client({

//We'lluseFacebookforthisexample

authEndpoint:'https://www.facebook.com/dialog/oauth',

tokenEndpoint:'https://graph.facebook.com/oauth/access_token',

activeUserEndpoint:'https://graph.facebook.com/v2.0/me',

clientId:'keyboardcat',

clientSecret:'keyboardcat'

module.context.use('/oauth2',router);

//Seetheusermanagementexampleforsettingupthe

//sessionsandusersobjectsusedinthisexample

router.use(sessions);

router.post('/auth',function(req,res){

constcsrfToken=crypto.genRandomAlphaNumbers(32);

consturl=req.reverse('oauth2_callback',{csrfToken});

constredirect_uri=req.makeAbsolute(url);

//SetCSRFcookieforfiveminutes

res.cookie('oauth2_csrf_token',csrfToken,{ttl:60*5});

//Redirecttotheprovider'sauthorizationURL

res.redirect(303,oauth2.getAuthUrl(redirect_uri));

router.get('/auth',function(req,res){

//Someproviderspasserrorsasqueryparameter

if(req.queryParams.error){

res.throw(500,`Providererror:${req.queryParams.error}`)

//MakesureCSRFcookiematchestheURL

constexpectedToken=req.cookie('oauth2_csrf_token');

if(!expectedToken||req.queryParams.csrfToken!==expectedToken){

res.throw(400,'CSRFmismatch.');

//MakesuretheURLcontainsagranttoken

if(!req.queryParams.code){

res.throw(400,'Providerdidnotpassgranttoken.');

//Reconstructtheredirect_uriusedforthegranttoken

consturl=req.reverse('oauth2_callback');

constredirect_uri=req.makeAbsolute(url);

//Fetchanaccesstokenfromtheprovider

constauthData=oauth2.exchangeGrantToken(

req.queryParams.code,

redirect_uri

constfacebookToken=authData.access_token;

//Fetchtheactiveuser'sprofileinfo

constprofile=oauth2.fetchActiveUser(facebookToken);

constfacebookId=profile.id;

//TrytofindanexistinguserwiththeuserID

//(thisrequirestheuserscollection)

letuser=users.firstExample({facebookId});

if(user){

//UpdatethefacebookTokenifithaschanged

if(user.facebookToken!==facebookToken){

users.update(user,{facebookToken});

}else{

OAuth2.0

//Createanewuserdocument

user={

username:`fb:${facebookId}`,

facebookId,

facebookToken

//Logtheuserin(thisrequiresthesessionmiddleware)

req.session.facebookToken=authData.facebookToken;

//Redirecttothedefaultroute

res.redirect(303,req.makeAbsolute('/'));

},'oauth2_callback')

.queryParam('error',joi.string().optional())

.queryParam('csrfToken',joi.string().optional())

.queryParam('code',joi.string().optional());

CreatinganOAuth2.0clientcreateOAuth2Client(options):OAuth2Client

CreatesanOAuth2.0client.

Arguments

options:Object

authEndpoint:string

Thefully-qualifiedURLoftheprovider'sauthorizationendpoint.

tokenEndpoint:string

Thefully-qualifiedURLoftheprovider'stokenendpoint.

refreshEndpoint:string(optional)

Thefully-qualifiedURLoftheprovider'srefreshtokenendpoint.

activeUserEndpoint:string(optional)

Thefully-qualifiedURLoftheprovider'sendpointforfetchingdetailsaboutthecurrentuser.

clientId:string

Theapplication'sClientID(orAppID)fortheprovider.

clientSecret:string

Theapplication'sClientSecret(orAppSecret)fortheprovider.

ReturnsanOAuth2.0clientforthegivenprovider.

SettingupOAuth2.0forFacebook

IfyouwanttouseFacebookastheOAuth2.0provider,usethefollowingoptions:

authEndpoint:https://www.facebook.com/dialog/oauthtokenEndpoint:https://graph.facebook.com/oauth/access_tokenactiveUserEndpoint:https://graph.facebook.com/v2.0/me

YoualsoneedtoobtainaclientIDandclientsecretfromFacebook:

1. CreatearegularaccountatFacebookoruseanexistingaccountyouown.2. VisittheFacebookDeveloperspage.

OAuth2.0

3. ClickonAppsinthemenu,thenselectRegisterasaDeveloper(theonlyoption)andfollowtheinstructionsprovided.Youmayneedtoverifyyouraccountbyphone.

4. ClickonAppsinthemenu,thenselectCreateaNewAppandfollowtheinstructionsprovided.5. Opentheappdashboard,thennotedowntheAppIDandAppSecret.Thesecretmaybehiddenbydefault.6. ClickonSettings,thenAdvancedandenteroneormoreValidOAuthredirectURIs.Atleastoneofthemmustmatchyourredirect_urilater.Don'tforgettosaveyourchanges.

7. SettheoptionclientIdtotheAppIDandtheoptionclientSecrettotheAppSecret.

SettingupOAuth2.0forGitHub

IfyouwanttouseGitHubastheOAuth2.0provider,usethefollowingoptions:

authEndpoint:https://github.com/login/oauth/authorize?scope=usertokenEndpoint:https://github.com/login/oauth/access_tokenactiveUserEndpoint:https://api.github.com/user

YoualsoneedtoobtainaclientIDandclientsecretfromGitHub:

1. CreatearegularaccountatGitHuboruseanexistingaccountyouown.2. GotoAccountSettings>Applications>Registernewapplication.3. ProvideanauthorizationcallbackURL.Thismustmatchyourredirect_urilater.4. Fillintheotherrequireddetailsandfollowtheinstructionsprovided.5. Opentheapplicationpage,thennotedowntheClientIDandClientSecret.6. SettheoptionclientIdtotheClientIDandtheoptionclientSecrettotheClientSecret.

SettingupOAuth2.0forGoogle

IfyouwanttouseGoogleastheOAuth2.0provider,usethefollowingoptions:

authEndpoint:https://accounts.google.com/o/oauth2/auth?access_type=offline&scope=profiletokenEndpoint:https://accounts.google.com/o/oauth2/tokenactiveUserEndpoint:https://www.googleapis.com/plus/v1/people/me

YoualsoneedtoobtainaclientIDandclientsecretfromGoogle:

1. CreatearegularaccountatGoogleoruseanexistingaccountyouown.2. VisittheGoogleDevelopersConsole.3. ClickonCreateProject,thenfollowtheinstructionsprovided.4. Whenyourprojectisready,opentheprojectdashboard,thenclickonEnableanAPI.5. EnabletheGoogle+APItoallowyourapptodistinguishbetweendifferentusers.6. OpentheCredentialspageandclickCreatenewClientID,thenfollowtheinstructionsprovided.AtleastoneAuthorizedRedirectURImustmatchyourredirect_urilater.AtleastoneAuthorizedJavaScriptOriginmustmatchyourapp'sfully-qualifieddomain.

7. WhentheClientIDisready,notedowntheClientIDandClientsecret.8. SettheoptionclientIdtotheClientIDandtheoptionclientSecrettotheClientsecret.

GettheauthorizationURLoauth2.getAuthUrl(redirect_uri,args):string

GeneratestheauthorizationURLfortheauthorizationendpoint.

Arguments

redirect_uri:string

Thefully-qualifiedURLofyourapplication'sOAuth2.0callback.

args:(optional)

response_type:string(Default:"code")

OAuth2.0

SeeRFC6749.

Returnsafully-qualifiedURLfortheauthorizationendpointoftheproviderbyappendingtheclientIDandanyadditionalargumentsfromargstotheauthEndpoint.

Exchangeagrantcodeforanaccesstokenoauth2.exchangeGrantToken(code,redirect_uri)

Exchangesagrantcodeforanaccesstoken.

PerformsaPOSTresponsetothetokenEndpoint.

Arguments

code:string

Agrantcodereturnedbytheprovider'sauthorizationendpoint.

redirect_uri:string

TheoriginalcallbackURLwithwhichthecodewasrequested.

args:Object(optional)

grant_type:string(Default:"authorization_code")

SeeRFC6749.

Fetchtheactiveuseroauth2.fetchActiveUser(access_token):Object

Fetchesdetailsoftheactiveuser.

PerformsaGETresponsetotheactiveUserEndpoint.

ReturnsnulliftheactiveUserEndpointisnotconfigured.

Arguments

access_token:string

AnOAuth2.0accesstokenasreturnedbyexchangeGrantToken.

Examples

constauthData=oauth2.exchangeGrantToken(code,redirect_uri);

constuserData=oauth2.fetchActiveUser(authData.access_token);

OAuth2.0

TransactionsStartingwithversion1.3,ArangoDBprovidessupportforuser-definabletransactions.

TransactionsinArangoDBareatomic,consistent,isolated,anddurable(ACID).

TheseACIDpropertiesprovidethefollowingguarantees:

Theatomicityprinciplemakestransactionseithercompleteintheirentiretyorhavenoeffectatall.Theconsistencyprincipleensuresthatnoconstraintsorotherinvariantswillbeviolatedduringorafteranytransaction.Theisolationpropertywillhidethemodificationsofatransactionfromothertransactionsuntilthetransactioncommits.Finally,thedurabilitypropositionmakessurethatoperationsfromtransactionsthathavecommittedwillbemadepersistent.TheamountoftransactiondurabilityisconfigurableinArangoDB,asisthedurabilityoncollectionlevel.

Transactions

TransactioninvocationArangoDBtransactionsaredifferentfromtransactionsinSQL.

InSQL,transactionsarestartedwithexplicitBEGINorSTARTTRANSACTIONcommand.Followinganyseriesofdataretrievalormodificationoperations,anSQLtransactionisfinishedwithaCOMMITcommand,orrolledbackwithaROLLBACKcommand.Theremaybeclient/servercommunicationbetweenthestartandthecommit/rollbackofanSQLtransaction.

InArangoDB,atransactionisalwaysaserver-sideoperation,andisexecutedontheserverinonego,withoutanyclientinteraction.Alloperationstobeexecutedinsideatransactionneedtobeknownbytheserverwhenthetransactionisstarted.

TherearenoindividualBEGIN,COMMITorROLLBACKtransactioncommandsinArangoDB.Instead,atransactioninArangoDBisstartedbyprovidingadescriptionofthetransactiontothedb._executeTransactionJavaScriptfunction:

db._executeTransaction(description);

Thisfunctionwillthenautomaticallystartatransaction,executeallrequireddataretrievaland/ormodificationoperations,andattheendautomaticallycommitthetransaction.Ifanerroroccursduringtransactionexecution,thetransactionisautomaticallyaborted,andallchangesarerolledback.

Executetransaction

executesatransactiondb._executeTransaction(object)

Executesaserver-sidetransaction,asspecifiedbyobject.

objectmusthavethefollowingattributes:

collections:asub-objectthatdefineswhichcollectionswillbeusedinthetransaction.collectionscanhavetheseattributes:read:asinglecollectionoralistofcollectionsthatwillbeusedinthetransactioninread-onlymodewrite:asinglecollectionoralistofcollectionsthatwillbeusedinthetransactioninwriteorreadmode.

action:aJavascriptfunctionorastringwithJavascriptcodecontainingalltheinstructionstobeexecutedinsidethetransaction.Ifthecoderunsthroughsuccessfully,thetransactionwillbecommittedattheend.Ifthecodethrowsanexception,thetransactionwillberolledbackandalldatabaseoperationswillberolledback.

Additionally,objectcanhavethefollowingoptionalattributes:

waitForSync:booleanflagindicatingwhetherthetransactionisforcedtobesynchronous.lockTimeout:anumericvaluethatcanbeusedtosetatimeoutforwaitingoncollectionlocks.Ifnotspecified,adefaultvaluewillbeused.SettinglockTimeoutto0willmakeArangoDBnottimeoutwaitingforalock.params:optionalargumentspassedtothefunctionspecifiedinaction.

ThefollowingattributescanbeusedfortransactionsintheRocksDBstorageengine:

maxTransactionSize:transactionsizelimitinbytesintermediateCommitSize:maximumtotalsizeofoperationsafterwhichanintermediatecommitisperformedautomaticallyintermediateCommitCount:maximumnumberofoperationsafterwhichanintermediatecommitisperformedautomatically

Declarationofcollections

Allcollectionswhicharetoparticipateinatransactionneedtobedeclaredbeforehand.Thisisanecessitytoensureproperlockingandisolation.

Collectionscanbeusedinatransactioninwritemodeorinread-onlymode.

Ifanydatamodificationoperationsaretobeexecuted,thecollectionmustbedeclaredforuseinwritemode.Thewritemodeallowsmodifyingandreadingdatafromthecollectionduringthetransaction(i.e.thewritemodeincludesthereadmode).

Contrary,usingacollectioninread-onlymodewillonlyallowperformingreadoperationsonacollection.Anyattempttowriteintoacollectionusedinread-onlymodewillmakethetransactionfail.

Collectionsforatransactionaredeclaredbyprovidingtheminthecollectionsattributeoftheobjectpassedtothe_executeTransactionfunction.Thecollectionsattributehasthesub-attributesreadandwrite:

db._executeTransaction({

collections:{

write:["users","logins"],

read:["recommendations"]

readandwriteareoptionalattributes,andonlyneedtobespecifiediftheoperationsinsidethetransactionsdemandforit.

Thecontentsofreadorwritecaneachbelistsarrayscollectionnamesorasinglecollectionname(asastring):

collections:{

write:"users",

read:"recommendations"

Note:Itiscurrentlyoptionaltospecifycollectionsforread-onlyaccess.Evenwithoutspecifyingthem,itisstillpossibletoreadfromsuchcollectionsfromwithinatransaction,butwithrelaxedisolation.PleaserefertoTransactionsLockingformoredetails.

Inordertomakeatransactionfailwhenanon-declaredcollectionisusedinsideforreading,theoptionalallowImplicitsub-attributeofcollectionscanbesettofalse:

collections:{

read:"recommendations",

allowImplicit:false/*thisdisallowsreadaccesstoothercollections

thanspecified*/

action:function(){

vardb=require("@arangodb").db;

returndb.foobar.toArray();/*willfailbecausedb.foobarmustnotbeaccessed

forreadinginsidethistransaction*/

ThedefaultvalueforallowImplicitistrue.Write-accessingcollectionsthathavenotbeendeclaredinthecollectionsarrayisneverpossible,regardlessofthevalueofallowImplicit.

Declarationofdatamodificationandretrievaloperations

AlldatamodificationandretrievaloperationsthataretobeexecutedinsidethetransactionneedtobespecifiedinaJavascriptfunction,usingtheactionattribute:

collections:{

write:"users"

action:function(){

//alloperationsgohere

AnyvalidJavascriptcodeisallowedinsideactionbutthecodemayonlyaccessthecollectionsdeclaredincollections.actionmaybeaJavascriptfunctionasshownabove,orastringrepresentationofaJavascriptfunction:

collections:{

write:"users"

action:"function(){doSomething();}"

Pleasenotethatanyoperationsspecifiedinactionwillbeexecutedontheserver,inaseparatescope.Variableswillbeboundlate.AccessinganyJavaScriptvariablesdefinedontheclient-sideorinsomeotherservercontextfrominsideatransactionmaynotwork.Instead,anyvariablesusedinsideactionshouldbedefinedinsideactionitself:

collections:{

write:"users"

action:function(){

vardb=require(...).db;

db.users.save({...});

Whenthecodeinsidetheactionattributeisexecuted,thetransactionisalreadystartedandallrequiredlockshavebeenacquired.Whenthecodeinsidetheactionattributefinishes,thetransactionwillautomaticallycommit.Thereisnoexplicitcommitcommand.

Tomakeatransactionabortandrollbackallchanges,anexceptionneedstobethrownandnotcaughtinsidethetransaction:

collections:{

write:"users"

action:function(){

db.users.save({_key:"hello"});

//willabortandrollbackthetransaction

throw"doh!";

Thereisnoexplicitabortorrollbackcommand.

Asmentionedearlier,atransactionwillcommitautomaticallywhentheendoftheactionfunctionisreachedandnoexceptionhasbeenthrown.Inthiscase,theusercanreturnanylegalJavaScriptvaluefromthefunction:

collections:{

write:"users"

action:function(){

db.users.save({_key:"hello"});

//willcommitthetransactionandreturnthevalue"hello"

return"hello";

Customexceptions

Onemaywishtodefinecustomexceptionsinsideofatransaction.Tohavetheexceptionpropagateupwardsproperly,pleasethrowananinstanceofbaseJavaScriptErrorclassoraderivative.Tospecifyanerrornumber,includeitastheerrorNumberfield.Asanexample:

collections:{},

action:function(){

varerr=newError('Myerrorcontext');

err.errorNumber=1234;

throwerr;

Note:Inpreviousversions,customexceptionswhichdidnothaveanError-likeformweresimplyconvertedtostringsandexposedintheexceptionfieldofthereturnederror.Thisisnolongerthecase,asithadthepotentialtoleakunwantedinformationifimproperlyused.

Examples

Thefirstexamplewillwrite3documentsintoacollectionnamedc1.Thec1collectionneedstobedeclaredinthewriteattributeofthecollectionsattributepassedtotheexecuteTransactionfunction.

Theactionattributecontainstheactualtransactioncodetobeexecuted.Thiscodecontainsalldatamodificationoperations(3inthisexample).

//setup

db._create("c1");

collections:{

write:["c1"]

action:function(){

db.c1.save({_key:"key1"});

db.c1.count();//3

Abortingthetransactionbythrowinganexceptionintheactionfunctionwillrevertallchanges,soasifthetransactionneverhappened:

//setup

db._create("c1");

collections:{

write:["c1"]

action:function(){

db.c1.count();//1

db.c1.count();//2

throw"doh!";

db.c1.count();//0

Theautomaticrollbackisalsoexecutedwhenaninternalexceptionisthrownatsomepointduringtransactionexecution:

//setup

db._create("c1");

collections:{

write:["c1"]

action:function(){

//willthrowduplicateakeyerror,notexplicitlyrequestedbytheuser

//we'llnevergethere...

db.c1.count();//0

Asrequiredbytheconsistencyprinciple,abortingorrollingbackatransactionwillalsorestoresecondaryindexestothestateattransactionstart.

Cross-collectiontransactions

There'salsothepossibilitytorunatransactionacrossmultiplecollections.Inthiscase,multiplecollectionsneedtobedeclaredinthecollectionsattribute,e.g.:

//setup

db._create("c1");

db._create("c2");

collections:{

write:["c1","c2"]

action:function(){

db.c1.count();//1

db.c2.count();//1

Again,throwinganexceptionfrominsidetheactionfunctionwillmakethetransactionabortandrollbackallchangesinallcollections:

//setup

db._create("c1");

db._create("c2");

collections:{

write:["c1","c2"]

action:function(){

for(vari=0;i<100;++i){

db.c1.save({_key:"key"+i});

db.c2.save({_key:"key"+i});

db.c1.count();//100

db.c2.count();//100

//abort

throw"doh!"

db.c1.count();//0

db.c2.count();//0

PassingparameterstotransactionsArbitraryparameterscanbepassedtotransactionsbysettingtheparamsattributewhendeclaringthetransaction.Thisfeatureishandytore-usethesametransactioncodeformultiplecallsbutwithdifferentparameters.

Abasicexample:

collections:{},

action:function(params){

returnparams[1];

params:[1,2,3]

Theaboveexamplewillreturn2.

Someexamplethatusescollections:

collections:{

write:"users",

read:["c1","c2"]

action:function(params){

vardb=require('@arangodb').db;

vardoc=db.c1.document(params['c1Key']);

db.users.save(doc);

doc=db.c2.document(params['c2Key']);

db.users.save(doc);

params:{

c1Key:"foo",

c2Key:"bar"

Passingparameters

LockingandIsolationTransactionsneedtospecifyfromwhichcollectionstheywillreaddataandwhichcollectionstheyintendtomodify.Thiscanbedonebysettingtheread,write,orexclusiveattributesinthecollectionsattributeofthetransaction:

collections:{

read:"users",

write:["test","log"]

action:function(){

constdb=require("@arangodb").db;

db.users.toArray().forEach(function(doc){

db.log.insert({value:"removeduser:"+doc.name});

db.test.remove(doc._key);

writeheremeanswriteaccesstothecollection,andalsoincludesanyreadaccesses.exclusiveisasynonymforwriteintheMMFilesengine,becausebothexclusiveandwritewillacquirecollection-levellocksinthisengine.IntheRocksDBengine,exclusivemeansexclusivewriteaccesstothecollection,andwritemeans(shared)writeaccesstothecollection,whichcanbeinterleavedwithwriteaccessesbyotherconcurrenttransactions.

MMFilesengine

TheMMFilesengineusesthefollowinglockingmechanismstoserializetransactionsonthesamedata:

Allcollectionsspecifiedinthecollectionsattributearelockedintherequestedmode(readorwrite)attransactionstart.Lockingofmultiplecollectionsisperformedinalphabeticalorder.Whenatransactioncommitsorrollsback,alllocksarereleasedinreverseorder.Thelockingorderisdeterministictoavoiddeadlocks.

Whilelocksareheld,modificationsbyothertransactionstothecollectionsparticipatinginthetransactionareprevented.Atransactionwillthusseeaconsistentviewoftheparticipatingcollections'data.

Additionally,atransactionwillnotbeinterruptedorinterleavedwithanyotherongoingoperationsonthesamecollection.Thismeanseachtransactionwillruninisolation.Atransactionshouldneverseeuncommittedorrolledbackmodificationsbyothertransactions.Additionally,readsinsideatransactionarerepeatable.

Notethattheaboveistrueonlyforallcollectionsthataredeclaredinthecollectionsattributeofthetransaction.

RocksDBengine

TheRocksDBenginedoesnotlockanycollectionsparticipatinginatransactionforread.Readoperationscanruninparalleltootherreadorwriteoperationsonthesamecollections.

Forallcollectionsthatareusedinwritemode,theRocksDBenginewillinternallyacquirea(shared)readlock.Thismeansthatmanywriterscanmodifydatainthesamecollectioninparallel(andalsoruninparalleltoongoingreads).However,iftwoconcurrenttransactionsattempttomodifythesamedocumentorindexentry,therewillbeawrite-writeconflict,andoneofthetransactionswillabortwitherror1200("conflict").Itisthenuptoclientapplicationstoretrythefailedtransactionoracceptthefailure.

Inordertoguardlong-runningorcomplextransactionsagainstconcurrentoperationsonthesamedata,theRocksDBengineallowstoaccesscollectionsinexclusivemode.Exclusiveaccesseswillinternallyacquireawrite-lockonthecollections,sotheyarenotexecutedinparallelwithanyotherwriteoperations.Readoperationscanstillbecarriedoutbyotherconcurrenttransactions.

Lazilyaddingcollections

Theremightbesituationswhendeclaringallcollectionsaprioriisnotpossible,forexample,becausefurthercollectionsaredeterminedbyadynamicAQLqueryinsidethetransaction,forexampleaqueryusingAQLgraphtraversal.

Lockingandisolation

Inthiscase,itwouldbeimpossibletoknowbeforehandwhichcollectiontolock,andthusitislegaltonotdeclarecollectionsthatwillbeaccessedinthetransactioninread-onlymode.Accessinganon-declaredcollectioninread-onlymodeduringatransactionwilladdthecollectiontothetransactionlazily,andfetchdatafromthecollectionasusual.However,asthecollectionisaddedlazily,thereisnoisolationfromotherconcurrentoperationsortransactions.Readsfromsuchcollectionsarepotentiallynon-repeatable.

Examples:

collections:{

read:"users"

action:function(){

/*ExecuteanAQLquerythattraversesagraphstartingata"users"vertex.

Itisyetunknownintowhichothercollectionsthequerymighttraverse*/

db._createStatement({

query:`FORvINANY"users/1234"connectionsRETURNv`

}).execute().toArray().forEach(function(d){

/*...*/

Thisautomaticlazyadditionofcollectionstoatransactionalsointroducesthepossibilityofdeadlocks.Deadlocksmayoccurifthereareconcurrenttransactionsthattrytoacquirelocksonthesamecollectionslazily.

Inordertomakeatransactionfailwhenanon-declaredcollectionisusedinsideatransactionforreading,theoptionalallowImplicitsub-attributeofcollectionscanbesettofalse:

collections:{

read:"users",

allowImplicit:false

action:function(){

/*Thebelowquerywillnowfailbecausethecollection"connections"hasnot

beenspecifiedinthelistofcollectionsusedbythetransaction*/

db._createStatement({

query:`FORvINANY"users/1234"connectionsRETURNv`

}).execute().toArray().forEach(function(d){

/*...*/

ThedefaultvalueforallowImplicitistrue.Write-accessingcollectionsthathavenotbeendeclaredinthecollectionsarrayisneverpossible,regardlessofthevalueofallowImplicit.

Ifusers/1234hasanedgeinconnections,linkingittoanotherdocumentintheuserscollection,thenthefollowingexplicitdeclarationwillwork:

collections:{

read:["users","connections"],

allowImplicit:false

/*...*/

Iftheedgepointstoadocumentinanothercollectionhowever,thenthequerywillfail,unlessthatothercollectionisaddedtothedeclarationaswell.

Notethatifadocumenthandleisusedasstartingpointforatraversal,e.g.FORvINANY"users/not_linked"...orFORvINANY{_id:"users/not_linked"}...,thennoerrorisraisedinthecaseofthestartvertexnothavinganyedgestofollow,withallowImplicit:falseandusersnotbeingdeclaredforreadaccess.AQLonlyseesastringanddoesnotconsideritareadaccess,unlessthereareedgesconnectedtoit.FORvINANYDOCUMENT("users/not_linked")...willfailevenwithoutedges,asitisalwaysconsideredtobeareadaccesstotheuserscollection.

Lockingandisolation

DeadlocksandDeadlockdetection

Adeadlockisasituationinwhichtwoormoreconcurrentoperations(usertransactionsorAQLqueries)trytoaccessthesameresources(collections,documents)andneedtowaitfortheotherstofinish,butnoneofthemcanmakeanyprogress.

AgoodexampleforadeadlockistwoconcurrentlyexecutingtransactionsT1andT2thattrytoaccessthesamecollectionsbutthatneedtowaitforeachother.Inthisexample,transactionT1willwritetocollectionc1,butwillalsoreaddocumentsfromcollectionc2withoutannouncingit:

collections:{

write:"c1"

action:function(){

/*writeintoc1(announced)*/

db.c1.insert({foo:"bar"});

/*someoperationherethattakeslongtoexecute...*/

/*readfromc2(unannounced)*/

db.c2.toArray();

TransactionT2announcestowriteintocollectionc2,butwillalsoreaddocumentsfromcollectionc1withoutannouncingit:

collections:{

write:"c2"

action:function(){

/*writeintoc2(announced)*/

db.c2.insert({bar:"baz"});

/*someoperationherethattakeslongtoexecute...*/

/*readfromc1(unannounced)*/

db.c1.toArray();

Intheaboveexample,adeadlockwilloccuriftransactionT1andT2havebothacquiredtheirwritelocks(T1forcollectionc1andT2forcollectionc2)andarethentryingtoreadfromtheotherother(T1willreadfromc2,T2willreadfromc1).T1willthentrytoacquirethereadlockoncollectionc2,whichispreventedbytransactionT2.T2howeverwillwaitforthereadlockoncollectionc1,whichispreventedbytransactionT1.

Incaseofsuchdeadlock,therewouldbenoprogressforanyoftheinvolvedtransactions,andnoneoftheinvolvedtransactionscouldevercomplete.Thisiscompletelyundesirable,sotheautomaticdeadlockdetectionmechanisminArangoDBwillautomaticallyabortoneofthetransactionsinvolvedinsuchdeadlock.Abortingmeansthatallchangesdonebythetransactionwillberolledbackanderror29(deadlockdetected)willbethrown.

Clientcode(AQLqueries,usertransactions)thataccessesmorethanonecollectionshouldbeawareofthepotentialofdeadlocksandshouldhandletheerror29(deadlockdetected)properly,eitherbypassingtheexceptiontothecallerorretryingtheoperation.

Toavoidbothdeadlocksandnon-repeatablereads,allcollectionsusedinatransactionshouldbespecifiedinthecollectionsattributewhenknowninadvance.Incasethisisnotpossiblebecausecollectionsareaddeddynamicallyinsidethetransaction,deadlocksmayoccurandthedeadlockdetectionmaykickinandabortthetransaction.

TheRocksDBengineusesdocument-levellocksandthereforewillnothaveadeadlockproblemoncollectionlevel.Iftwoconcurrenttransactionshowevermodifythesamedocumentsorindexentries,theRocksDBenginewillsignalawrite-writeconflictandabortoneofthetransactionswitherror1200("conflict")automatically.

Lockingandisolation

DurabilityTransactionsareexecutedinmainmemoryfirstuntilthereiseitherarollbackoracommit.Onrollback,nodatawillbewrittentodisk,buttheoperationsfromthetransactionwillbereversedinmemory.

Oncommit,allmodificationsdoneinthetransactionwillbewrittentothecollectiondatafiles.ThesewriteswillbesynchronizedtodiskifanyofthemodifiedcollectionshasthewaitForSyncpropertysettotrue,orifanyindividualoperationinthetransactionwasexecutedwiththewaitForSyncattribute.Additionally,transactionsthatmodifydatainmorethanonecollectionareautomaticallysynchronizedtodisk.Thissynchronizationisdonetonotonlyensuredurability,buttoalsoensureconsistencyincaseofaservercrash.

Thatmeansifyouonlymodifydatainasinglecollection,andthatcollectionhasitswaitForSyncpropertysettofalse,thewholetransactionwillnotbesynchronizedtodiskinstantly,butwithasmalldelay.

Thereisthusthepotentialriskoflosingdatabetweenthecommitofthetransactionandtheactual(delayed)disksynchronization.ThisisthesameaswritingintocollectionsthathavethewaitForSyncpropertysettofalseoutsideofatransaction.IncaseofacrashwithwaitForSyncsettofalse,theoperationsperformedinthetransactionwilleitherbevisiblecompletelyornotatall,dependingonwhetherthedelayedsynchronizationhadkickedinornot.

ToensuredurabilityoftransactionsonacollectionthathavethewaitForSyncpropertysettofalse,youcansetthewaitForSyncattributeoftheobjectthatispassedtoexecuteTransaction.ThiswillforceasynchronizationofthetransactiontodiskevenforcollectionsthathavewaitForSyncsettofalse:

collections:{

write:"users"

waitForSync:true,

action:function(){...}

Analternativeistoperformanoperationwithanexplicitsyncrequestinatransaction,e.g.

db.users.save({_key:"1234"},true);

Inthiscase,thetruevaluewillmakethewholetransactionbesynchronizedtodiskatthecommit.

Inanycase,ArangoDBwillgiveusersthechoiceofwhetherornottheywantfulldurabilityforsinglecollectiontransactions.Usingthedelayedsynchronization(i.e.waitForSyncwithavalueoffalse)willpotentiallyincreasethroughputandperformanceoftransactions,butwillintroducetheriskoflosingthelastcommittedtransactionsinthecaseofacrash.

Incontrast,transactionsthatmodifydatainmorethanonecollectionareautomaticallysynchronizedtodisk.Thiscomesatthecostofseveraldisksync.Foramulti-collectiontransaction,thecalltothe_executeTransactionfunctionwillonlyreturnafterthedataofallmodifiedcollectionshasbeensynchronizedtodiskandthetransactionhasbeenmadefullydurable.Thisnotonlyreducestheriskoflosingdataincaseofacrashbutalsoensuresconsistencyafterarestart.

Incaseofaservercrash,anymulti-collectiontransactionsthatwerenotyetcommittedorinpreparationtobecommittedwillberolledbackonserverrestart.

Formulti-collectiontransactions,therewillbeatleastonedisksyncoperationpermodifiedcollection.Multi-collectiontransactionsthushaveapotentiallyhighercostthansinglecollectiontransactions.Thereisnoconfigurationtoturnoffdisksynchronizationformulti-collectiontransactionsinArangoDB.Thedisksyncspeedofthesystemwillthusbethemostimportantfactorfortheperformanceofmulti-collectiontransactions.

Durability

Limitations

InGeneral

TransactionsinArangoDBhavebeendesignedwithparticularusecasesinmind.Theywillbemainlyusefulforshortandsmalldataretrievaland/ormodificationoperations.

Theimplementationisnotoptimizedforverylong-runningorveryvoluminousoperations,andmaynotbeusableforthesecases.

Onelimitationisthatatransactionoperationinformationmustfitintomainmemory.Thetransactioninformationconsistsofrecordpointers,revisionnumbersandrollbackinformation.Theactualdatamodificationoperationsofatransactionarewrittentothewrite-aheadloganddonotneedtofitentirelyintomainmemory.

Ongoingtransactionswillalsopreventthewrite-aheadlogsfrombeingfullygarbage-collected.Informationinthewrite-aheadlogfilescannotbewrittentocollectiondatafilesorbediscardedwhiletransactionsareongoing.

Toensureprogressofthewrite-aheadloggarbagecollection,transactionsshouldbekeptassmallaspossible,andbigtransactionsshouldbesplitintomultiplesmallertransactions.

TransactionsinArangoDBcannotbenested,i.e.atransactionmustnotstartanothertransaction.Ifanattemptismadetocallatransactionfrominsidearunningtransaction,theserverwillthrowerror1651(nestedtransactionsdetected).

ItisalsodisallowedtoexecuteusertransactiononsomeofArangoDB'sownsystemcollections.Thisshouldn'tbeaproblemforregularusageassystemcollectionswillnotcontainuserdataandthereisnoneedtoaccessthemfromwithinausertransaction.

Someoperationsarenotallowedinsidetransactionsingeneral:

creationanddeletionofdatabases(db._createDatabase(),db._dropDatabase())creationanddeletionofcollections(db._create(),db._drop(),db.<collection>.rename())creationanddeletionofindexes(db.<collection>.ensureIndex(),db.<collection>.dropIndex())

Ifanattemptismadetocarryoutanyoftheseoperationsduringatransaction,ArangoDBwillabortthetransactionwitherrorcode1653(disallowedoperationinsidetransaction).

Finally,allcollectionsthatmaybemodifiedduringatransactionmustbedeclaredbeforehand,i.e.usingthecollectionsattributeoftheobjectpassedtothe_executeTransactionfunction.Ifanyattemptismadetocarryoutadatamodificationoperationonacollectionthatwasnotdeclaredinthecollectionsattribute,thetransactionwillbeabortedandArangoDBwillthrowerror1652unregisteredcollectionusedintransaction.Itislegaltonotdeclareread-onlycollections,butthisshouldbeavoidedifpossibletoreducetheprobabilityofdeadlocksandnon-repeatablereads.

PleaserefertoLockingandIsolationformoredetails.

InClustersUsingasingleinstanceofArangoDB,multi-document/multi-collectionqueriesareguaranteedtobefullyACID.ThisismorethanmanyotherNoSQLdatabasesystemssupport.Inclustermode,single-documentoperationsarealsofullyACID.Multi-document/multi-collectionqueriesinaclusterarenotACID,whichisequallythecasewithcompetingdatabasesystems.TransactionsinaclusterwillbesupportedinafutureversionofArangoDBandmaketheseoperationsfullyACIDaswell.Notethatfornon-shardedcollectionsinacluster,thetransactionalpropertiesofasingleserverapply(fullyACID).

TransactionsintheRocksDBstorageengineDataofongoingtransactionsisstoredinRAM.Transactionsthatgettoobig(intermsofnumberofoperationsinvolvedorthetotalsizeofdatacreatedormodifiedbythetransaction)willbecommittedautomatically.EffectivelythismeansthatbigusertransactionsaresplitintomultiplesmallerRocksDBtransactionsthatarecommittedindividually.TheentireusertransactionwillnotnecessarilyhaveACIDpropertiesinthiscase.

ThefollowingglobaloptionscanbeusedtocontroltheRAMusageandautomaticintermediatecommitsfortheRocksDBengine:

Limitations

--rocksdb.max-transaction-size

Transactionsizelimit(inbytes).TransactionsstoreallkeysandvaluesinRAM,solargetransactionsruntheriskofcausingout-of-memorysitations.Thissettingallowsyoutoensurethatdoesnothappenbylimitingthesizeofanyindividualtransaction.TransactionswhoseoperationswouldconsumemoreRAMthanthisthresholdvaluewillabortautomaticallywitherror32("resourcelimitexceeded").

--rocksdb.intermediate-commit-size

Ifthesizeofalloperationsinatransactionreachesthisthreshold,thetransactioniscommittedautomaticallyandanewtransactionisstarted.Thevalueisspecifiedinbytes.

--rocksdb.intermediate-commit-count

Ifthenumberofoperationsinatransactionreachesthisvalue,thetransactioniscommittedautomaticallyandanewtransactionisstarted.

Theabovevaluescanalsobeadjustedpertransaction,bysettingthefollowingattributesinthecalltodb._executeTransaction():

maxTransactionSize:transactionsizelimitinbytesintermediateCommitSize:maximumtotalsizeofoperationsafterwhichanintermediatecommitisperformedautomaticallyintermediateCommitCount:maximumnumberofoperationsafterwhichanintermediatecommitisperformedautomatically

Limitations

DeploymentInthischapterwedescribevariouspossibilitiestodeployArangoDB.Inparticularfortheclustermode,therearedifferentwaysandwewanttohighlighttheiradvantagesanddisadvantages.Weevendocumentindetail,howtosetupaclusterbysimplystartingvariousArangoDBprocessesondifferentmachines,eitherdirectlyorusingDockercontainers.

SingleinstanceCluster:DC/OS,ApacheMesosandMarathonCluster:Generic&DockerMultipleDatacentersAdvancedTopics

Cluster:TestsetuponalocalmachineCluster:StartingprocessesondifferentmachinesCluster:LaunchinganArangoDBclusterusingDockercontainersAgency

Deployment

Singleinstancedeployment

ThelatestofficialbuildsofArangoDBforallsupportedoperatingsystemsmaybeobtainedfromhttps://www.arangodb.com/download/.

Linuxremarks

Besidestheofficialimageswhichareprovidedforthemostpopularlinuxdistributionstherearealsoavarietyofunofficialimagesprovidedbythecommunity.Wearetrackingmostofthecommunitycontributions(includingneworupdatedimages)inournewsletter:

https://www.arangodb.com/category/newsletter/

Windowsremarks

PleasenotethatArangoDBwillonlyworkon64bit.

Docker

ThesimplestwaytodeployArangoDBisusingDocker.TogetageneralunderstandingofDockerhavealookattheirexcellentdocumentation.

Authentication

TostarttheofficialDockercontaineryouwillhavetodecideonanauthenticationmethod.Otherwisethecontainerwon'tstart.

ProvideoneoftheargumentstoDockerasanenvironmentvariable.

Therearethreeoptions:

1. ARANGO_NO_AUTH=1

Disableauthenticationcompletely.Usefulforlocaltestingorforoperatinginatrustednetwork(withoutapublicinterface).

2. ARANGO_ROOT_PASSWORD=password

StartArangoDBwiththegivenpasswordforroot

3. ARANGO_RANDOM_ROOT_PASSWORD=1

LetArangoDBgeneratearandomrootpassword

Togetgoingquickly:

dockerrun-eARANGO_RANDOM_ROOT_PASSWORD=1arangodb/arangodb

ForanindepthguideaboutDockerandArangoDBpleasechecktheofficialdocumentation:https://hub.docker.com/r/arangodb/arangodb/.Notethatweareusingtheimagearangodb/arangodbherewhichisalwaysthemostcurrentone.Thereisalsothe"official"onecalledarangodbwhosedocumentationishere:https://hub.docker.com/_/arangodb/

Singleinstance

Cluster

DistributeddeploymentusingApacheMesos

ArangoDBhasasophisticatedandyeteasywaytouseclustermode.Toleveragethefullclusterfeatureset(monitoring,scaling,automaticfailoverandautomaticreplacementoffailednodes)youhavetorunArangoDBonsomekindofclustermanagementsystem.CurrentlyArangoDBreliesonApacheMesosinthatmatter.Mesosisaclusteroperatingsystemwhichpowerssomeoftheworldsbiggestdatacentersrunningseveralthousandsofnodes.

DC/OSistherecommendedwaytoinstallaclusterasiteasesmuchoftheprocesstoinstallaMesoscluster.YoucandeployitveryquicklyonavarietyofcloudhostersorsetupyourownDC/OSlocally.DC/OSisasetoftoolsbuiltontopofApacheMesos.ApacheMesosisasocalled"DistributedClusterOperationSystem"andthecoreofDC/OS.ApacheMesoshastheconceptofsocalledpersistentvolumeswhichmakeitperfectlysuitableforadatabase.

Installing

FirstprepareaDC/OSclusterbygoingtohttps://dcos.ioandfollowingtheinstructionsthere.

DC/OScomeswithitsownpackagemanagement.Packagescanbeinstalledfromthesocalled"Universe".AsanofficialDC/OSpartnerArangoDBcanbeinstalledfromtherestraightaway.

1. InstallingviaDC/OSUI

i. OpenyourbrowserandgototheDC/OSadmininterfaceii. Openthe"Universe"tabiii. Locatearangodbandhit"InstallPackage"iv. Press"InstallPackage"

2. InstallingviatheDC/OScommandline

i. Installthedcoscliii. Openaterminalandissuedcosinstallarangodb

Bothoptionsareessentiallydoingthesameinthebackground.BotharestartingArangoDBwithitsdefaultoptionsset.

Toreviewthedefaultoptionsusingthewebinterfacesimplyclick"AdvancedInstallation"inthewebinterface.Thereyouwillfindalistofoptionsincludingsomeexplanation.

ToreviewthedefaultoptionsusingtheCLIfirsttypedcospackagedescribe--configarangodb.Thiswillgiveyouaflatlistofdefaultsettings.

Togetanexplanationofthevariouscommandlineoptionspleasecheckthelatestoptionshere(choosethemostrecentnumberandhavealookatconfig.json):

https://github.com/mesosphere/universe/tree/version-3.x/repo/packages/A/arangodb

AfterinstallationDC/OSwillstartdeployingtheArangoDBclusterontheDC/OScluster.YoucanwatchArangoDBstartingonthe"Services"tabinthewebinterface.OnceitislistedashealthyclickthelinknexttoitandyoushouldseetheArangoDBwebinterface.

ArangoDBMesosframework

AssoonasArangoDBwasdeployedMesoswillkeepyourclusterrunning.ThewebinterfacehasmanymonitoringfacilitiessobesuretomakeyourselffamiliarwiththeDC/OSwebinterface.AsafaulttolerantsystemMesoswilltakecareofmostfailurescenariosautomatically.MesosdoesthatbyrunningArangoDBasasocalled"framework".ThisframeworkhasbeenspecificallybuilttokeepArangoDBrunninginahealthyconditionontheMesoscluster.Fromtimetotimeataskmightfail.TheArangoDBframeworkwillthentakecareofreschedulingthefailedtask.Asitknowsabouttheveryspecificsofeachclustertaskanditsroleitwillautomaticallytakecareofmostfailurescenarios.

Toinspectwhattheframeworkisdoinggotohttp://web-interface-url/mesosinyourbrowser.Locatethetask"arangodb"andinspectstderrinthe"Sandbox".Thiscanbeofinterestforexamplewhenaslavegotlostandtheframeworkisreschedulingthetask.

Cluster:Mesos,DC/OS

UsingArangoDB

TouseArangoDBasadatastoreinyourDC/OSclusteryoucanfacilitatetheservicediscoveryofDC/OS.AssumingyoudeployedastandardArangoDBclusterthemesosdnswillknowaboutarangodb.mesos.BydoingaSRVDNSrequest(checkthedocumentationofmesosdns)youcanfindouttheportwheretheinternalHAProxyofArangoDBisrunning.ThiswillofferaroundrobinloadbalancertoaccessallArangoDBcoordinators.

ScalingArangoDB

TochangethesettingsofyourArangoDBClusteraccesstheArangoDBUIandhit"Nodes".Onthescaletabyouwillhavetheabilitytoscaleyourclusterupanddown.

AfterchangingthesettingstheArangoDBframeworkwilltakecareoftherest.ScalingyourclusterupisgenerallyastraightforwardoperationasMesoswillsimplylaunchanothertaskandbedonewithit.Scalingdownisabitmorecomplicatedasthedatafirsthastobemovedtosomeotherplacesothatwillnaturallytakesomewhatlonger.

Pleasenotethatscalingoperationsmightnotalwayswork.ForexampleiftheunderlyingMesosclusteriscompletelysaturatedwithitsrunningtasksscalingupwillnotbepossible.ScalingdownmightalsofailduetotheclusternotbeingabletomoveallshardsofaDBServertoanewdestinationbecauseofsizelimitations.BesuretochecktheoutputoftheArangoDBframework.

Deinstallation

DeinstallingArangoDBisabitmoredifficultasthereismuchstatebeingkeptintheMesosclusterwhichisnotautomaticallycleanedup.Todeinstallfromthecommandlineusethefollowingoneliner:

dcosarangodbuninstall;dcospackageuninstallarangodb

Thiswillfirstcleanupthestateintheclusterandthenuninstallarangodb.

arangodb-cleanup-framework

Shouldyouforgettocleanupthestateyoucandosolaterbyusingthearangodb-cleanup-frameworkcontainer.Otherwiseyoumightnotbeabletodeployanewarangodbinstallation.

ThecleanupframeworkwillannounceitselfasanormalArangoDB.MesoswillrecognizethisandofferallpersistentvolumesitstillhasforArangoDBtothisframework.Thecleanupframeworkwillthenproperlyfreethepersistentvolumes.Finallyitwillcleanupanystateleftinzookeeper(thecentralconfigurationmanagerinaMesoscluster).

Todeploythecleanupframework,followtheinstructionsinthegithubrepository.AfterdeploymentwatchtheoutputinthesandboxoftheMesoswebinterface.Afterawhilethereshouldn'tbeanypersistentresourceoffersanymoreaseverythingwascleanedup.AfterthatyoucandeletethecleanupframeworkagainviaMarathon.

ApacheMesosandMarathon

YoucanalsoinstallArangoDBonabareApacheMesosclusterprovidedthatMarathonisrunningonit.

Doingsohasthefollowingdownsides:

1. ManualMesosclustersetup2. Youneedtoimplementyourownservicediscovery3. Youaremissingthedcoscli4. Installationanddeinstallationaretedious5. YouneedtosetupsomekindofproxytunneltoaccessArangoDBfromtheoutside6. Sparsemonitoringcapabilities

HoweverthesearethingswhichdonotinfluenceArangoDBitselfandoperatingyourclusterlikethisisfullysupported.

InstallingviaMarathon

ToinstallArangoDBviamarathonyouneedaproperconfigfile:

Cluster:Mesos,DC/OS

"id":"arangodb",

"cpus":0.25,

"mem":256.0,

"ports":[0,0,0],

"instances":1,

"args":[

"framework",

"--framework_name=arangodb",

"--master=zk://172.17.0.2:2181/mesos",

"--zk=zk://172.17.0.2:2181/arangodb",

"--user=",

"--principal=pri",

"--role=arangodb",

"--mode=cluster",

"--async_replication=true",

"--minimal_resources_agent=mem(*):512;cpus(*):0.25;disk(*):512",

"--minimal_resources_dbserver=mem(*):512;cpus(*):0.25;disk(*):1024",

"--minimal_resources_secondary=mem(*):512;cpus(*):0.25;disk(*):1024",

"--minimal_resources_coordinator=mem(*):512;cpus(*):0.25;disk(*):1024",

"--nr_agents=1",

"--nr_dbservers=2",

"--nr_coordinators=2",

"--failover_timeout=86400",

"--arangodb_image=arangodb/arangodb-mesos:3.1",

"--secondaries_with_dbservers=false",

"--coordinators_with_dbservers=false"

"container":{

"type":"DOCKER",

"docker":{

"image":"arangodb/arangodb-mesos-framework:3.1",

"network":"HOST"

"healthChecks":[

"protocol":"HTTP",

"path":"/framework/v1/health.json",

"gracePeriodSeconds":3,

"intervalSeconds":10,

"portIndex":0,

"timeoutSeconds":10,

"maxConsecutiveFailures":0

Carefullyreviewthesettings(especiallytheIPsandtheresources).ThenyoucandeploytoMarathon:

curl-XPOST-H"Content-Type:application/json"http://url-of-marathon/v2/apps-d@arangodb3.json

AlternativelyusethewebinterfaceofMarathontodeployArangoDB.IthasaJSONmodeandyoucanusetheaboveconfigurationfile.

DeinstallationviaMarathon

AswithDC/OSyoufirstneedtoproperlycleanupanystateleftovers.

TheeasiestistosimplydeleteArangoDBandthendeploythecleanup-framework(seesectionarangodb-cleanup-framework).

Configurationoptions

TheArangodbMesosframeworkhasatonofdifferentoptionswhicharelistedanddescribedhere:https://github.com/arangodb/arangodb-mesos-framework/tree/3.1

Cluster:Mesos,DC/OS

AutomaticnativeClusters

SimilarlytohowtheMesosframeworkarangesanArangoDBclusterinaDC/OSenvironmentforyou,arangodbcandothisforyouinaplainenvironment.

Byinvokingthefirstarangodbyoulaunchaprimarynode.Itwillbindanetworkport,andoutputthecommandsyouneedtocut'n'pasteintotheothernodes.Let'sreviewtheprocessofsuchastartuponthreehostsnamedh01,h02,andh03:

arangodb@h01~>arangodb--ownAddressh01:4000

2017/06/1214:59:38Startingarangodbversion0.5.0+git,build5f97368

2017/06/1214:59:38ServingasmasterwithID'52698769'onh01:4000...

2017/06/1214:59:38Waitingfor3serverstoshowup.

2017/06/1214:59:38Usethefollowingcommandstostartotherservers:

arangodb--dataDir=./db2--joinh01:4000

arangodb--dataDir=./db3--joinh01:4000

2017/06/1214:59:38Listeningon0.0.0.0:4000(h01:4000)

Soyoucutthelinesarangodb--data.dir=./db2--starter.join127.0.0.1andexecutethemfortheothernodes.Ifyourunitonanothernodeonyournetwork,replacethe--starter.join127.0.0.1bythepublicIPofthefirsthost.

arangodbh02~>arangodb--dataDir=./db2--joinh01:4000

2017/06/1214:48:50Contactingmasterh01:4000...

2017/06/1214:48:50Waitingfor3serverstoshowup...

2017/06/1214:48:50Listeningon0.0.0.0:4000(:4000)

arangodbh03~>arangodb--dataDir=./db3--joinh01:4000

2017/06/1214:48:50Contactingmasterh01:4000...

2017/06/1214:48:50Waitingfor3serverstoshowup...

2017/06/1214:48:50Listeningon0.0.0.0:4000(:4000)

Oncethetwootherprocessesjoinedthecluster,andstartedtheirArangoDBserverprocesses(thismaytakeawhiledependingonyoursystem),itwillinformyouwheretoconnecttheClusterfromaBrowser,shelloryourprogramm:

2017/06/1214:55:21coordinatorupandrunning.

Atthispointyoumayaccessyourclusterateithercoordinatorendpoint,http://h01:4002/,http://h02:4002/orhttp://h03:4002/.

AutomaticnativelocaltestClusters

Ifyouonlywantalocaltestcluster,youcanrunasinglestarterwiththe--starter.localargument.Itwillstarta3"machine"clusteronyourlocalPC.

arangodb--starter.local

Note.AlocalclusterisintendedonlyfortestpurposessinceafailureofasinglePCwillbringdowntheentirecluster.

AutomaticDockerClusters

ArangoDBStartercanalsobeusedtolaunchclustersbasedondockercontainers.Itsabitmorecomplicated,sinceyouneedtoprovideinformationaboutyourenvironmentthatcan'tbeautodetected.

IntheDockerworldyouneedtotakecareaboutwherepersistantdataisstored,sincecontainersareintendedtobevolatile.Weuseavolumenamedarangodb1here:

dockervolumecreatearangodb1

(Youcanuseanytypeofdockervolumethatfitsyoursetupinstead.)

WethenneedtodeterminethetheIPofthedockerhostwhereyouintendtorunArangoDBstarteron.Dependingonyouroperatingsystemexecuteipaddr,ifconfigoripconfigtodetermineyourlocalipaddress.

192.168.1.0/24deveth0protokernelscopelinksrc192.168.1.32

SothisexampleusestheIP192.168.1.32:

dockerrun-it--name=adb1--rm-p8528:8528\

-varangodb1:/data\

-v/var/run/docker.sock:/var/run/docker.sock\

arangodb/arangodb-starter\

--starter.address=192.168.1.32

Itwillstartthemasterinstance,andcommandyoutostarttheslaveinstances:

Unabletofindimage'arangodb/arangodb-starter:latest'locally

latest:Pullingfromarangodb/arangodb-starter

Digest:sha256:b87d20c0b4757b7daa4cb7a9f55cb130c90a09ddfd0366a91970bcf31a7fd5a4

Status:Downloadednewerimageforarangodb/arangodb-starter:latest

2017/06/1213:26:14Startingarangodbversion0.7.1,buildf128884

2017/06/1213:26:14ServingasmasterwithID'46a2b40d'on192.168.1.32:8528...

2017/06/1213:26:14Waitingfor3serverstoshowup.

2017/06/1213:26:14Usethefollowingcommandstostartotherservers:

dockervolumecreatearangodb2&&\

dockerrun-it--name=adb2--rm-p8533:8528-varangodb2:/data\

-v/var/run/docker.sock:/var/run/docker.sockarangodb/arangodb-starter:0.7\

--starter.address=192.168.1.32--starter.join=192.168.1.32

dockervolumecreatearangodb3&&\

dockerrun-it--name=adb3--rm-p8538:8528-varangodb3:/data\

-v/var/run/docker.sock:/var/run/docker.sockarangodb/arangodb-starter:0.7\

--starter.address=192.168.1.32--starter.join=192.168.1.32

Onceyoustarttheotherinstances,itwillcontinuelikethis:

2017/05/1109:05:45Addedmaster'fc673b3b':192.168.1.32,portOffset:0

2017/05/1109:05:45Addednewpeer'e98ea757':192.168.1.32,portOffset:5

2017/05/1109:05:50Addednewpeer'eb01d0ef':192.168.1.32,portOffset:10

2017/05/1109:05:51Startingservice...

2017/05/1109:05:51Lookingforarunninginstanceofagentonport8531

2017/05/1109:05:51Startingagentonport8531

2017/05/1109:05:52Lookingforarunninginstanceofdbserveronport8530

2017/05/1109:05:52Startingdbserveronport8530

2017/05/1109:05:53Lookingforarunninginstanceofcoordinatoronport8529

2017/05/1109:05:53Startingcoordinatoronport8529

2017/05/1109:05:58agentupandrunning(version3.2.devel).

2017/05/1109:06:15dbserverupandrunning(version3.2.devel).

2017/05/1109:06:31coordinatorupandrunning(version3.2.devel).

Andatleastittellsyouwhereyoucanworkwithyourcluster:

2017/05/1109:06:31Yourclustercannowbeaccessedwithabrowserat`http://192.168.1.32:8529`or

2017/05/1109:06:31using`arangosh--server.endpointtcp://192.168.1.32:8529`.

Underthehood

Thefirstarangodbyouran(asshownabove)willbecomethemasterinyoursetup,the--starter.joinwillbetheslaves.

ThemasterdetermineswhichArangoDBserverprocessestolaunchonwhichslave,andhowtheyshouldcommunicate.Itwillthenlaunchtheserverprocessesandmonitorthem.Onceithasdetectedthatthesetupiscompleteyouwillgettheprompt.Themasterwillsavethesetupforsubsequentstarts.

MorecomplicatedsetupoptionscanbefoundinArangoDBStartersReadme.

AdvancedTopics

Incontrasttotheothertopicsinthischapterthatstrivetogetyousimplysetupinpreparedenvironments,Thefollowingchaptersdescribewhatsgoingonunderthehoodindetails,thecomponentsofArangoDBClusters,andhowthey'reputtogether:

RunningalocaltestsetupRunningadistributedsetupRunninginDocker

AdvancedTopics

LaunchingArangoDB'sstandalone"agency"

MultipleArangoDBinstancescanbedeployedasafault-tolerantdistributedstatemachine.

Whatisafault-tolerantstatemachineinthefirstplace?

Inmanyservicedeploymentsconsistingofarbitrarycomponentsdistributedovermultiplemachinesoneisfacedwiththechallengeofcreatingadependablecentralisedknowledgebaseorconfiguration.Implementationofsuchaserviceturnsouttobeoneofthemostfundamentalproblemsininformationengineering.Whileitmayseemasiftherealisationofsuchaserviceiseasilyconceivable,dependablityformulatesaparadoxononcomputernetworksperse.Ontheonehand,oneneedsadistributedsystemtoavoidasinglepointoffailure.Ontheotherhand,onehastoestablishconsensusamongthecomputersinvolved.

Consensusisthekeywordhereanditsrealisationonanetworkprovestobefarfromtrivial.Manypapersandconferenceproceedingshavediscussedandevaluatedthiskeychallenge.Twoalgorithms,historicallyfarapart,havebecomewidelypopular,namelyPaxosanditsderivativesandRaft.Discussingthemandtheirdifferences,althoughhighlyenjoyable,mustremainfarbeyondthescopeofthisdocument.Findthereferencestothemainpublicationsatthebottomofthispage.

AtArangoDB,wedecidedtoimplementRaftasitisarguablytheeasiertounderstandandthusimplement.Insimpleterms,Raftguaranteesthatalinearstreamoftransactions,isreplicatedinrealtimeamongagroupofmachinesthroughanelectedleader,whointurnmusthaveaccesstoandprojectleadershipuponanoverallmajorityofparticipatinginstances.InArangoDBweliketocalltheentiretyofthecomponentsofthereplicatedtransactionlog,thatisthemachinesandtheArangoDBinstances,whichconstitutethereplicatedlog,theagency.

Startup

Theagencymustconsistsofanoddnumberofagentsinordertobeabletoestablishanoverallmajorityandsomemeansfortheagentstobeabletofindoneanotheratstartup.

Themostobviouswaywouldbetoinformallagentsoftheaddressesandportsoftherest.Thishowever,ismoreinformationthanneeded.Forexample,itwouldsuffice,ifallagentswouldknowtheaddressandportofthenextagentinacyclicfashion.Anotherstraitforwardsolutionwouldbetoinformallagentsoftheaddressandportofsaythefirstagent.

Clearlyallcases,whichwouldformdisjunctsubsetsofagentswouldbreakorintheleastimpairthefunctionalityoftheagency.Fromthereontheagentswillgossipthemissinginformationabouttheirpeers.

Typically,oneachievesfairlyhighfault-tolerancewithlow,oddnumberofagentswhilekeepingthenecessarynetworktrafficataminimum.Itseemsthatthetypicalagencysizewillbeinrangeof3to7agents.

Thebelowcommandsstartupa3-hostagencyononephysical/logicalboxwithports8529,8530and8531fordemonstrationpurposes.Theadressofthefirstinstance,port8529,isknowntotheothertwo.Afteratmost2roundsofgossipping,thelast2agentswillhaveacompletepictureoftheirsurroundingandpersistitforthenextrestart.

./build/bin/arangod--agency.activatetrue--agency.size3--agency.my-addresstcp://localhost:8529--server.authenticationfal

se--server.endpointtcp://0.0.0.0:8529agency-8529

./build/bin/arangod--agency.activatetrue--agency.size3--agency.endpointtcp://localhost:8529--agency.my-addresstcp://loc

alhost:8530--server.authenticationfalse--server.endpointtcp://0.0.0.0:8530agency-8530

./build/bin/arangod--agency.activatetrue--agency.size3--agency.endpointtcp://localhost:8529--agency.my-addresstcp://loc

alhost:8531--server.authenticationfalse--server.endpointtcp://0.0.0.0:8531agency-8531

Theparameteragency.endpointisthekeyingredientforthesecondandthirdinstancestofindthefirstinstanceandthusformacompleteagency.Pleaserefertothetheshell-scriptscripts/startStandaloneAgency.shongithuborinthesourcedirectory.

Key-value-storeAPI

Theagencyshouldbeupandrunningwithinacoupleofseconds,duringwhichtheinstanceshavegossipedtheirwayintoknowingtheotheragentsandelectedaleader.ThepublicAPIcanbecheckedforthestateoftheconfiguration:

curl-slocalhost:8529/_api/agency/config

AdvancedTopics

"term":1,

"leaderId":"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98",

"lastCommitted":1,

"lastAcked":{

"ac129027-b440-4c4f-84e9-75c042942171":0.21,

"c54dbb8a-723d-4c82-98de-8c841a14a112":0.21,

"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98":0

"configuration":{

"pool":{

"ac129027-b440-4c4f-84e9-75c042942171":"tcp://localhost:8531",

"c54dbb8a-723d-4c82-98de-8c841a14a112":"tcp://localhost:8530",

"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98":"tcp://localhost:8529"

"active":[

"ac129027-b440-4c4f-84e9-75c042942171",

"c54dbb8a-723d-4c82-98de-8c841a14a112",

"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98"

"id":"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98",

"agencysize":3,

"poolsize":3,

"endpoint":"tcp://localhost:8529",

"minping":0.5,

"maxping":2.5,

"supervision":false,

"supervisionfrequency":5,

"compactionstepsize":1000,

"supervisiongraceperiod":120

Tohighlightsomedetailsintheaboveoutputlookfor"term"and"leaderId".BotharekeyinformationaboutthecurrentstateoftheRaftalgorithm.Youmayhavenotedthatthefirstelectiontermhasestablishedarandomleaderfortheagency,whoisinchargeofreplicationofthestatemachineandforallexternalreadandwriterequestsuntilsuchtimethattheprocessgetsisolatedfromtheothertwosubsequenctlylosingitsleadership.

ReadandWriteAPIs

Generally,allreadandwriteaccessesaretransactionsmoreoveranyreadandwriteaccessmayconsistofmultiplesuchtransactionsformulatedasarraysofarraysinJSONdocuments.

Readtransaction

Anagencystartedfromscratchwilldealwiththesimplestqueryasfollows:

curl-Llocalhost:8529/_api/agency/read-d'[["/"]]'

Theaboverequestforanemptykeyvaluestorewillreturnwithanemptydocument.Theinnerarraybracketswillaggregatearesultfrommultiplesourcesinthekey-value-storewhiletheouterarraywilldelivermultiplesuchaggregatedresults.Alsonotethe-Lcurlflag,whichallowstherequesttofollowredirectstothecurrentleader.

Considerthefollowingkey-value-store:

"baz":12,

"corge":{

"e":2.718281828459045,

"pi":3.14159265359

"foo":{

"bar":"HelloWorld"

AdvancedTopics

"qux":{

"quux":"HelloWorld"

Thefollowingarrayofreadtransactionswillyield:

curl-Llocalhost:8529/_api/agency/read-d'[["/foo","/foo/bar","/baz"],["/qux"]]'

"baz":12,

"foo":{

"bar":"HelloWorld"

"qux":{

"quux":"HelloWorld"

Notethattheresultisanarrayoftworesultsforthefirstandsecondreadtransactionsfromaboveaccordingly.Alsonotethattheresultsfromthefirstreadtransactionareaggregatedinto

"baz":12,

"foo":{

"bar":"HelloWorld"

Theaggregationisperformedon2levels:

1. /foo/bariseliminatedasasubsetof/foo2. Theresultsfrom/fooand/bararejoined

Thewordtransactionmeansherethatitisguaranteedthatallaggregationshappeninquasi-realtimeandthatnowriteaccesscouldhavehappenedinbetween.

Btw,thesametransactiononthevirginkey-valuestorewouldproduce[{},{}]

WriteAPI:

ThewriteAPImustunfortunatelybealittlemorecomplex.MultipleroadsleadtoRome:

curl-Llocalhost:8529/_api/agency/write-d'[[{"/foo":{"op":"push","new":"bar"}}]]'

curl-Llocalhost:8529/_api/agency/write-d'[[{"/foo":{"op":"push","new":"baz"}}]]'

curl-Llocalhost:8529/_api/agency/write-d'[[{"/foo":{"op":"push","new":"qux"}}]]'

curl-Llocalhost:8529/_api/agency/write-d'[[{"foo":["bar","baz","qux"]}]]'

areequivalentforexampleandwillcreateandfillanarrayat/foo.Here,again,theoutermostarrayisthecontainerforthetransactionarrays.

WedocumententacompleteguideoftheAPIintheAPIsection.

AdvancedTopics

LaunchinganArangoDBclusterfortesting

AnArangoDBclusterconsistsofseveralrunningtasks(orserverprocesses)whichformthecluster.ArangoDBitselfwon'tstartormonitoranyofthesetasks.Soitwillneedsomekindofsupervisorwhichismonitoringandstartingthesetasks.ForproductionusagewerecommendusingApacheMesosastheclustersupervisor.

HoweverstartingaclustermanuallyispossibleandisaveryeasymethodtogetafirstimpressionofwhatanArangoDBclusterlookslike.

Theeasiestwaytostartalocalclusterfortestingpurposesistorunscripts/startLocalCluster.shfromacloneofthesourcerepositoryaftercompilingArangoDBfromsource(seeinstructionsinthefileREADME_maintainers.mdintherepository.Thiswillstart1Agency,2DBServersand1Coordinator.Tostoptheclusterissuescripts/stopLocalCluster.sh.

ThissectionwilldiscusstherequiredparametersforeveryroleinanArangoDBcluster.BesuretoreadtheArchitecturedocumentationtogetabasicunderstandingoftheunderlyingarchitectureandtheinvolvedrolesinanArangoDBcluster.

Inthefollowingsectionswewillgothroughtherelevantoptionsperrole.

Agency

Tostartupanagencyyoufirsthavetoactivateit.Thisisdonebyproviding--agency.activatetrue.

Tostartuptheagencyinitsfaulttolerantmodesetthe--agency.sizeto3.Youwillthenhavetoprovideatleast3agentsbeforetheagencywillstartoperation.

Duringinitializationtheagentshavetofindeachother.Todosoprovideatleastonecommon--agency.endpoint.Theagentswillthencoordinatestartupthemselves.Theywillannouncethemselveswiththeirexternaladdresswhichmaybespecifiedusing--agency.my-address.ThisisrequiredinbridgeddockersetupsorNATedenvironments.

Soinsummarythisiswhatyourstartupmightlooklike:

AdvancedTopics

arangod--server.endpointtcp://0.0.0.0:5001--agency.my-address=tcp://127.0.0.1:5001--server.authenticationfalse--agency.ac

tivatetrue--agency.size3--agency.endpointtcp://127.0.0.1:5001--agency.supervisiontrue--database.directoryagency1&

Ifyouarehappywithasingleagent,thensimplyuseasinglecommandlikethis:

arangod--server.endpointtcp://0.0.0.0:5001--server.authenticationfalse--agency.activatetrue--agency.size1--agency.endp

ointtcp://127.0.0.1:5001--agency.supervisiontrue--database-directoryagency1&

Furthermore,inthefollowingsectionswhen--cluster.agency-addressisusedmultipletimestospecifyallthreeagentaddresses,justuseasingleoption--cluster.agency.addresstcp://127.0.0.1:5001instead.

CoordinatorsandDBServers

Thesetworolesshareacommonsetofrelevantoptions.Firstyoushouldspecifytheroleusing--cluster.my-role.ThiscaneitherbePRIMARY(adatabaseserver)orCOORDINATOR.Furthermoreprovidetheexternalendpoint(IPandport)ofthetaskvia--cluster.my-address.

Thefollowingisafull-exampleofwhatitmightlooklike:

arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8529--cluster.my-addresstcp://127.0.0.1:8529--cluster.

my-rolePRIMARY--cluster.agency-endpointtcp://127.0.0.1:5001--cluster.agency-endpointtcp://127.0.0.1:5002--cluster.agency-

endpointtcp://127.0.0.1:5003--database.directoryprimary1&

my-rolePRIMARY--cluster.agency-endpointtcp://127.0.0.1:5001--cluster.agency-endpointtcp://127.0.0.1:5002--cluster.agency-

endpointtcp://127.0.0.1:5003--database.directoryprimary2&

my-roleCOORDINATOR--cluster.agency-endpointtcp://127.0.0.1:5001--cluster.agency-endpointtcp://127.0.0.1:5002--cluster.age

ncy-endpointtcp://127.0.0.1:5003--database.directorycoordinator&

Noteinparticularthattheendpointdescriptionsgivenunder--cluster.my-addressand--cluster.agency-endpointmustnotusetheIPaddress0.0.0.0becausetheymustcontainanactualaddressthatcanberoutedtothecorrespondingserver.The0.0.0.0in--server.endpointsimplymeansthattheserverbindsitselftoallavailablenetworkdeviceswithallavailableIPaddresses.

UponregisteringwiththeagencyduringstartuptheclusterwillassignanIDtoeveryserver.ThegeneratedIDwillbeprintedouttothelogorcanbeaccessedviathehttpAPIbycallinghttp://server-address/_admin/server/id.

YouhavenowlaunchedacompleteArangoDBclusterandcancontactitscoordinatorattheendpointtcp://127.0.0.1:8531,whichmeansthatyoucanreachthewebUIunderhttp://127.0.0.1:8531.

AdvancedTopics

LaunchinganArangoDBclusteronmultiplemachines

Essentially,onecanusethemethodfromtheprevioussectiontostartanArangoDBclusteronmultiplemachinesaswell.Theonlychangesarethatonehastoreplacealllocaladdresses127.0.0.1bytheactualIPaddressofthecorrespondingserver.

IfweassumethatyouwanttostartyouArangoDBclusteronthreedifferentmachineswithIPaddresses

192.168.1.1

192.168.1.2

192.168.1.3

thenthecommandsyouhavetouseare(youcanusehostnamesiftheycanberesolvedtoIPaddressesonallmachines):

On192.168.1.1:

sudoarangod--server.endpointtcp://0.0.0.0:5001--agency.my-addresstcp://192.168.1.1:5001--server.authenticationfalse--ag

ency.activatetrue--agency.size3--agency.supervisiontrue--database.directoryagency

On192.168.1.2:

ency.activatetrue--agency.size3--agency.supervisiontrue--database.directoryagency

On192.168.1.3:

ency.activatetrue--agency.size3--agency.endpointtcp://192.168.1.1:5001--agency.endpointtcp://192.168.1.2:5001--agency.e

ndpointtcp://192.168.1.3:5001--agency.supervisiontrue--database.directoryagency

On192.168.1.1:

sudoarangod--server.authentication=false--server.endpointtcp://0.0.0.0:8529--cluster.my-addresstcp://192.168.1.1:8529--c

luster.my-rolePRIMARY--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agency-endpointtcp://192.168.1.2:5001--clus

ter.agency-endpointtcp://192.168.1.3:5001--database.directoryprimary1&

On192.168.1.2:

sudoarangod--server.authentication=false--server.endpointtcp://0.0.0.0:8530--cluster.my-addresstcp://192.168.1.2:8530--c

luster.my-rolePRIMARY--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agency-endpointtcp://192.168.1.2:5001--clus

ter.agency-endpointtcp://192.168.1.3:5001--database.directoryprimary2&

On192.168.1.3:

arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8531--cluster.my-addresstcp://192.168.1.3:8531--cluste

r.my-roleCOORDINATOR--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agency-endpointtcp://192.168.1.2:5001--clust

er.agency-endpointtcp://192.168.1.3:5001--database.directorycoordinator&

Obviously,itwouldnolongerbenecessarytousedifferentportnumbersondifferentservers.Wehavechosentokeepallportnumbersincomparisontothelocalsetuptominimizethenecessarychanges.

Afterhavingswallowedtheselongishcommands,wehopethatyouappreciatethesimplicityofthesetupwithApacheMesosandDC/OS.

AdvancedTopics

ArangoDBClusterandDocker

Networking

AbitofextracarehastobeinvestedduetothewayinwhichDockerisolatesitsnetwork.Bydefaultitfullyisolatesthenetworkandbydoingsoanendpointlike--server.endpointtcp://0.0.0.0:8529willonlybindtoallinterfacesinsidetheDockercontainerwhichdoesnotincludeanyexternalinterfaceonthehostmachine.ThismaybesufficientifyoujustwanttoaccessitlocallybutincaseyouwanttoexposeittotheoutsideyoumustfacilitateDockersportforwardingusingthe-pcommandlineoption.BesuretochecktheofficialDockerdocumentation.

Tosimplymakearangodbavailableonallhostinterfacesonport8529:

dockerrun-p8529:8529-eARANGO_NO_AUTH=1arangodb

AnotherpossibilityistostartDockervianetworkmodehost.Thisispossiblebutgenerallynotrecommended.TodoitanywaychecktheDockerdocumentationfordetails.

DockerandClustertasks

TostarttheclusterviaDockerisbasicallythesameasstartinglocallyoronmultiplemachines.Howeverjustlikewiththesinglenetworkingimagewewillfacenetworkingissues.Youcansimplyusethe-pflagtomaketheindividualtaskavailableonthehostmachineoryoucoulduseDocker'slinkstoenabletaskintercommunication.

PleasenotethattherearesomeflagsthatspecifyhowArangoDBcanreachataskfromtheoutside.Theseareveryimportantandbuiltforthisexactusecase.Anexampleconfigurationmightlooklikethis:

dockerrun-eARANGO_NO_AUTH=1-p192.168.1.1:10000:8529arangodb/arangodbarangod--server.endpointtcp://0.0.0.0:8529--clust

er.my-addresstcp://192.168.1.1:10000--cluster.my-rolePRIMARY--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agen

cy-endpointtcp://192.168.1.2:5002--cluster.agency-endpointtcp://192.168.1.3:5003

ThiswillstartaprimaryDBserverwithinaDockercontainerwithanisolatednetwork.WithintheDockercontaineritwillbindtoallinterfaces(thiswillbe127.0.0.1:8529andsomeinternalDockeriponport8529).Bysupplying-p192.168.1.1:10000:8529weareestablishingaportforwardingfromourlocalIP(192.168.1.1port10000inthisexample)toport8529insidethecontainer.Withinthecommandwearetellingarangodhowitcanbereachedfromtheoutside--cluster.my-addresstcp://192.168.1.1:10000.ThisinformationwillbeforwardedtotheagencysothattheothertasksinyourclustercanseehowthisparticularDBServermaybereached.

AdvancedTopics

DatacentertodatacenterreplicationdeploymentThischapterdescribeshowtodeployallthecomponentsneededfordatacentertodatacenterreplication.

Forageneralintroductiontodatacentertodatacenterreplication,pleaserefertotheDatacentertodatacenterreplicationchapter.

Requirementscanbefoundinthissection.

Deploymentsteps:

ClusterKafka&ZookeeperArangoSyncMasterArangoSyncWorkersPrometheus&Grafana(optional)

MultipleDatacenters

ArangoDBclusterThereareseveralwaystostartanArangoDBcluster.InthissectionwewillfocusonourrecommendedwaytostartArangoDB:theArangoDBStarter.

Datacentertodatacenterreplicationrequirestherocksdbstorageengine.Theexamplesetupdescribedinthissectionwillhaverocksdbenabled.Ifyouchoosetodeploywithadifferentstrategykeepinmindtosetthestorageengine.

FortheotherpossibilitiestodeployanArangoDBclusterpleaserefertothissection.

TheStartersimplifiesthingsfortheoperatorandwillcoordinateadistributedclusterstartupacrossseveralmachinesandassignclusterrolesautomatically.

Whenstartedonseveralmachinesandenoughmachineshavejoined,theStarterswillstartAgents,sCoordinatorsandDBserversonthesemachines.

WhenrunningtheStarterwillsuperviseitschildtasks(namelyCoordinators,DBserversandAgents)andrestartthemincaseoffailure.

Tostarttheclusterusingasystemdunitfileusethefollowing:

[Unit]

Description=RuntheArangoDBStarter

After=network.target

[Service]

Restart=on-failure

EnvironmentFile=/etc/arangodb.env

EnvironmentFile=/etc/arangodb.env.local

Environment=DATADIR=/var/lib/arangodb/cluster

ExecStartPre=/usr/bin/sh-c"mkdir-p${DATADIR}"

ExecStart=/usr/bin/arangodb\

--starter.address=${PRIVATEIP}\

--starter.data-dir=${DATADIR}\

--starter.join=${STARTERENDPOINTS}\

--server.storage-engine=rocksdb\

--auth.jwt-secret=${CLUSTERSECRETPATH}

TimeoutStopSec=60

[Install]

WantedBy=multi-user.target

Notethatwesetrocksdbintheunitservicefile.

Clusterauthentication

Thecommunicationbetweentheclusternodesuseatoken(JWT)toauthenticate.Thismustbesharedbetweenclusternodes.

Sharingsecretsisobviouslyaverydelicatetopic.Theaboveworkflowassumesthattheoperatorwillputasecretinafilenamed${CLUSTERSECRETPATH}.

WerecommendtouseadedicatedsystemformanagingsecretslikeHashiCorps'VaultorthesecretmanagementofDC/OS.

Requiredports

Assoonasenoughmachineshavejoined,theStarterwillbeginstartingAgents,CoordinatorsandDBservers.

Eachofthesetasksneedsaporttocommunicate.Pleasemakesurethatthefollowingportsareavailableonallmachines:

8529forCoordinators8530forDBservers8531forAgents

TheStarteritselfwilluseport8528.

Cluster

Recommendeddeploymentenvironment

SincetheAgentsaresocriticaltotheavailabilityofboththeArangoDBandtheArangoSynccluster,itisrecommendedtorunAgentsondedicatedmachines.Considerthesemachines"pets".

CoordinatorsandDBServerscanbedeployedofothermachinesthatshouldbeconsidered"cattle".

Cluster

Kafka&ZookeeperHowtodeployzookeeperHowtodeploykafkaAccessibleports

SincethekafkabrokersarereallyCPUandmemoryintensive,itisrecommendedtorunzookeeper&kakfaondedicatedmachines.

Considerthesemachines"pets".

Kafka&Zookeeper

ArangoSyncMasterTheArangoSyncMasterisresponsibleformanagingallsynchronization,creatingtasksandassigningthosetotheArangoSyncWorkers.Atleast2instancesmutsbedeployedineachdatacenter.Oneinstancewillbethe"leader",theotherwillbeaninactiveslave.Whentheleaderisgoneforashortwhile,oneoftheotherinstanceswilltakeover.

Withclustersofasignificantsize,thesyncmasterwillrequireasignificantsetofresources.Thereforeitisrecommendedtodeploythesyncmastersontheirownservers,equipedwithsufficientCPUpowerandmemorycapacity.

TostartanArangoSyncMasterusingasystemdservice,useaunitlikethis:

[Unit]

Description=RunArangoSyncinmastermode

[Service]

Restart=on-failure

LimitNOFILE=8192

ExecStart=/usr/sbin/arangosyncrunmaster\

--log.level=debug\

--cluster.endpoint=${CLUSTERENDPOINTS}\

--cluster.jwtSecret=${CLUSTERSECRET}\

--server.keyfile=${CERTIFICATEDIR}/tls.keyfile\

--server.client-cafile=${CERTIFICATEDIR}/client-auth-ca.crt\

--server.endpoint=https://${PUBLICIP}:${MASTERPORT}\

--server.port=${MASTERPORT}\

--master.jwtSecret=${MASTERSECRET}\

--mq.type=kafka\

--mq.kafka-addr=${KAFKAENDPOINTS}\

--mq.kafka-client-keyfile=${CERTIFICATEDIR}/kafka-client.key\

--mq.kafka-cacert=${CERTIFICATEDIR}/tls-ca.crt\

TimeoutStopSec=60

[Install]

ThesyncmasterneedsaTLSservercertificateandaIfyouwanttheservicetocreateaTLScertificate&clientauthenticationcertificate,forauthenticatingwithArangoSyncMastersinanotherdatacenter,foreverystart,addthistotheServicesection.

ExecStartPre=/usr/bin/sh-c"mkdir-p${CERTIFICATEDIR}"

ExecStartPre=/usr/sbin/arangosynccreatetlskeyfile\

--cacert=${CERTIFICATEDIR}/tls-ca.crt\

--cakey=${CERTIFICATEDIR}/tls-ca.key\

--keyfile=${CERTIFICATEDIR}/tls.keyfile\

--host=${PUBLICIP}\

--host=${PRIVATEIP}\

--host=${HOST}

ExecStartPre=/usr/sbin/arangosynccreateclient-authkeyfile\

--cacert=${CERTIFICATEDIR}/tls-ca.crt\

--cakey=${CERTIFICATEDIR}/tls-ca.key\

--keyfile=${CERTIFICATEDIR}/kafka-client.key\

--host=${PUBLICIP}\

--host=${PRIVATEIP}\

--host=${HOST}

TheArangoSyncMastermustbereachableonaTCPport${MASTERPORT}(usedwith--server.portoption).Thisportmustbereachablefrominsidethedatacenter(bysyncworkersandoperations)andfrominsideoftheotherdatacenter(bysyncmastersintheotherdatacenter).

ArangoSyncMaster

SincethesyncmasterscanbeCPUintensivewhenrunninglotsofdatabases&collections,itisrecommendedtorunthemondedicatedmachineswithalotofCPUpower.

Considerthesemachines"pets".

ArangoSyncMaster

ArangoSyncWorkersTheArangoSyncWorkerisresponsibleforexecutingsynchronizationtasks.Foroptimalperformanceatleast1workerinstancemustbeplacedoneverymachinethathasanArangoDBDBserverrunning.Thisensuresthattaskscanbeexecutedwithminimalnetworktrafficoutsideofthemachine.

SincesyncworkerswillautomaticallystoponcetheirTLSservercertificateexpires(whichissetto2yearsbydefault),itisrecommendedtorunatleast2instancesofaworkeroneverymachineinthedatacenter.Thatway,taskscanstillbeassignedinthemostoptimalway,evenwhenaworkerintemporarilydownforarestart.

TostartanArangoSyncWorkerusingasystemdservice,useaunitlikethis:

[Unit]

Description=RunArangoSyncinworkermode

[Service]

Restart=on-failure

Environment=PORT=8729

LimitNOFILE=1000000

ExecStart=/usr/sbin/arangosyncrunworker\

--log.level=debug\

--server.port=${PORT}\

--server.endpoint=https://${PRIVATEIP}:${PORT}\

--master.endpoint=${MASTERENDPOINTS}\

--master.jwtSecret=${MASTERSECRET}

TimeoutStopSec=60

[Install]

TheArangoSyncWorkermustbereachableonaTCPport${PORT}(usedwith--server.portoption).Thisportmustbereachablefrominsidethedatacenter(bysyncmasters).

Notethelargefiledescriptorlimit.Thesyncworkerrequiresabout30filedescriptorspershard.Ifyouusehardwarewithhugeresources,andstillrunoutoffiledescriptors,youcandecidetorunmultiplesyncworkersoneachmachineinordertospreadthetasksacrossthem.

RecommendeddeploymentenvironmentThesyncworkersshouldberunonallmachinesthatalsocontainanArangoDBDBServer.Thesyncworkercanbememoryintensivewhenrunninglotsofdatabases&collections.

Considerthesemachines"cattle".

ArangoSyncWorkers

Prometheus&Grafana(optional)ArangoSyncprovidesmetricsinaformatsupportedbyPrometheus.WealsoprovideastandardsetofdashboardsforviewingthosemetricsinGrafana.

Ifyouwanttousethesetools,pleaserefertotheirwebsitesforinstructionsonhowtodeploythem.

Afterdeployment,youmustconfigurePrometheususingaconfigurationfilethatinstructsitaboutwhichtargetstoscrape.ForArangoSyncyoushouldconfigurescrapetargetsforallsyncmastersandallsyncworkers.Todoso,youcanuseaconfigurationsuchasthis:

global:

scrape_interval:10s#scrapetargetsevery10seconds.

scrape_configs:

#Scrapsyncmasters

-job_name:'sync_master'

scheme:'https'

bearer_token:"${MONITORINGTOKEN}"

tls_config:

insecure_skip_verify:true

static_configs:

-targets:

-"${IPMASTERA1}:8629"

-"${IPMASTERA2}:8629"

-"${IPMASTERB1}:8629"

-"${IPMASTERB2}:8629"

labels:

type:"master"

relabel_configs:

-source_labels:[__address__]

regex:${IPMASTERA1}\:8629|${IPMASTERA2}\:8629

target_label:dc

replacement:A

regex:${IPMASTERB1}\:8629|${IPMASTERB2}\:8629

target_label:dc

replacement:B

regex:${IPMASTERA1}\:8629|${IPMASTERB1}\:8629

target_label:instance

replacement:1

regex:${IPMASTERA2}\:8629|${IPMASTERB2}\:8629

replacement:2

#Scrapsyncworkers

-job_name:'sync_worker'

scheme:'https'

bearer_token:"${MONITORINGTOKEN}"

tls_config:

insecure_skip_verify:true

static_configs:

-targets:

-"${IPWORKERA1}:8729"

-"${IPWORKERA2}:8729"

-"${IPWORKERB1}:8729"

-"${IPWORKERB2}:8729"

labels:

type:"worker"

relabel_configs:

regex:${IPWORKERA1}\:8729|${IPWORKERA2}\:8729

target_label:dc

replacement:A

regex:${IPWORKERB1}\:8729|${IPWORKERB2}\:8729

target_label:dc

replacement:B

Prometheus&Grafana

regex:${IPWORKERA1}\:8729|${IPWORKERB1}\:8729

replacement:1

regex:${IPWORKERA2}\:8729|${IPWORKERB2}\:8729

replacement:2

Note:Theaboveexampleassumes2datacenters,with2syncmasters&2syncworkersperdatacenter.Youhavetoreplaceall${...}variablesintheaboveconfigurationwithapplicablevaluesfromyourenvironment.

Prometheuscanbeamemory&CPUintensiveprocess.ItisrecommendedtokeepthemonothermachinesthanusedtoruntheArangoDBclusterorArangoSynccomponents.

Considerthesemachines"cattle",unlessyouconfigurealertingonprometheus,inwhichcaseitisrecommendedtoconsiderthesemachines"pets".

Prometheus&Grafana

AdministrationMostadministrationcanbemanagedusingthearangosh.

FilesystemsAsonewouldexpectforadatabase,werecommendalocallymountedfilesystems.

NFSorsimilarnetworkfilesystemswillnotwork.

OnLinuxwerecommendtheuseofext4fs,onWindowsNTFSandonMacOSHFS+.

WerecommendtonotuseBTRFSonLinux.ItisknowntonotworkwellinconjunctionwithArangoDB.WeexperiencedthatArangoDBfaceslatencyissuesonaccessingitsdatabasefilesonBTRFSpartitions.InconjunctionwithBTRFSandAUFSwealsosawdatalossonrestart.

Administration

WebInterfaceArangoDBcomeswithabuilt-inwebinterfaceforadministration.Theinterfacediffersforstandaloneinstancesandclustersetups.

Standalone:

Cluster:

WebInterface

DashboardTheDashboardtabprovidesstatisticswhicharepolledregularlyfromtheArangoDBserver.

RequestsStatistics:

RequestspersecondRequesttypesNumberofclientconnectionsTransfersizeTransfersize(distribution)AveragerequesttimeAveragerequesttime(distribution)

SystemResources:

NumberofthreadsMemoryVirtualsizeMajorpagefaultsUsedCPUtime

Replication:

ReplicationstateTotalsTicksProgress

Dashboard

ClusterTheclustersectiondisplaysstatisticsaboutthegeneralclusterperformance.

Statistics:

AvailableandmissingcoordinatorsAvailableandmissingdatabaseserversMemoryusage(percent)CurrentconnectionsData(bytes)HTTP(bytes)Averagerequesttime(seconds)

Overview

Theoverviewshowsavailableandmissingcoordinatorsanddatabaseservers.

Cluster

Functions:

CoordinatorDashboard:ClickonaCoordinatorwillopenastatisticsdashboard.

Information(Coordinator/Databaseservers):

NameEndpointLastHeartbeatStatusHealth

Shards

Theshardsectiondisplaysallavailableshardedcollections.

Cluster

Functions:

MoveShardLeader:Clickonaleaderdatabaseofashardserverwillopenamovesharddialog.Shardscanbetransferredtoallavailabledatabasservers,excepttheleadingdatabaseserveroranavailablefollower.MoveShardFollower:Clickonafollowerdatabaseofashardwillopenamovesharddialog.Shardscanbetransferredtoallavailabledatabasservers,excepttheleadingdatabaseserveroranavailablefollower.RebalanceShards:Anewdatabaseserverwillnothaveanyshards.Withtherebalancefunctionalitytheclusterwillstarttorebalanceshardsincludingemptydatabaseservers.

Information(collection):

ShardLeader(greenstate:synciscomplete)Followers

Cluster

CollectionsThecollectionssectiondisplaysallavailablecollections.Fromhereyoucancreatenewcollectionsandjumpintoacollectionfordetails(clickonacollectiontile).

Functions:

A:TogglefilterpropertiesB:SearchcollectionbynameD:CreatecollectionC:FilterpropertiesH:Showcollectiondetails(clicktile)

Information:

E:CollectiontypeF:Collectionstate(unloaded,loaded,...)G:Collectionname

Collection

Collections

Therearefourviewcategories:

1. Content:

CreateadocumentDeleteadocumentFilterdocumentsDownloaddocumentsUploaddocuments

2. Indices:

CreateindicesDeleteindices

3. Info:

Detailedcollectioninformationandstatistics4. Settings:

Configurename,journalsize,indexbuckets,waitforsyncDeletecollectionTruncatecollectionUnload/LoadcollectionSavemodifedproperties(name,journalsize,indexbuckets,waitforsync)

Additionalinformation:

Uploadformat:

I.Line-wise

{"_key":"key1",...}

{"_key":"key2",...}

II.JSONdocumentsinalist

{"_key":"key1",...},

{"_key":"key2",...}

Collections

DocumentThedocumentsectionoffersaeditorwhichletyoueditdocumentsandedgesofacollection.

Functions:

EditdocumentSavedocumentDeletedocmentSwitchbetweenTree/Code-ModeCreateanewdocument

Information:

Displays:_id,_rev,_keyproperties

Document

QueryViewThequeryviewoffersyouthreedifferentsubviews:

EditorRunningQueriesSlowQueryHistory

AQLQueryEditor

ThewebinterfaceoffersaAQLQueryEditor:

Theeditorissplitintotwoparts,thequeryeditorpaneandthebindparameterpane.

Theleftpaneisyourregularqueryinputfield,whereyoucaneditandthenexecuteorexplainyourqueries.Bydefault,theenteredbindparameterwillautomaticallyberecognizedandshowninthebindparametertableintherightpane,whereyoucaneasilyeditthem.

Theinputfieldsareequippedwithtypedetection.Thismeansyoudon'thavetousequotemarksaroundstring,justwritethemas-is.Numberswillbetreatedasnumbers,trueandfalseasbooleans,nullasnull-typevalue.Squarebracketscanbeusedtodefinearrays,andcurlybracesforobjects(keysandvalueshavetobesurroundedbydoublequotes).Thiswillmostlybewhatyouwant.Butifyouwanttoforcesomethingtobetreatedasstring,usequotationmarksforthevalue:

123//interpretedasnumber

"123"//interpretedasstring

["foo","bar",123,true]//interpretedasarray

['foo','bar',123,true]//interpretedasstring

Queries

IfyouareusedtoworkwithJSON,youmaywanttoswitchthebindparametereditortoJSONmodebyclickingontheupperrighttogglebutton.YoucantheneditthebindparametersinrawJSONformat.

CustomQueries

TosavethecurrentqueryusetheSavebuttoninthetopleftcorneroftheeditororusetheshortcut(seebelow).

BypressingtheQueriesbuttoninthetopleftcorneroftheeditoryouactivatethecustomqueriesview.Hereyoucanselectapreviouslystoredcustomqueryoroneofourqueryexamples.

Clickonaquerytitletogetacodepreview.Inaddition,thereareactionbuttonsto:

CopytoeditorExplainqueryRunqueryDeletequery

Forthebuilt-inexamplequeries,thereisonlyCopytoeditoravailable.

ToexportorimportqueriestoandfromJSONyoucanusethebuttonsontheright-handside.

Result

Queries

Eachqueryyouexecuteorexplainopensupanewresultbox,soyouareabletofireupmultiplequeriesandviewtheirresultsatthesametime.Everyqueryresultboxgivesyoudetailedqueryinformationandofcoursethequeryresultitself.Theresultboxescanbedismissedindividually,oraltogetherusingtheRemoveresultsbutton.ThetogglebuttoninthetoprightcornerofeachboxswitchesbackandforthbetweentheResultandAQLquerywithbindparameters.

Spotlight

Thespotlightfeatureopensupamodalview.ThereyoucanfindallAQLkeywords,AQLfunctionsandcollections(filteredbytheirtype)tohelpyoutobemoreproductiveinwritingyourqueries.Spotlightcanbeopenedbythemagicwandiconinthetoolbarorviashortcut(seebelow).

AQLEditorShortcuts

Queries

Ctrl/Cmd+ReturntoexecuteaqueryCtrl/Cmd+Shift+ReturntoexplainaqueryCtrl/Cmd+Shift+StosavethecurrentqueryCtrl/Cmd+Shift+CtotogglecommentsCtrl+SpacetoopenupthespotlightsearchCtrl+Cmd+ZtoundolastchangeCtrl+Cmd+Shift+Ztoredolastchange

RunningQueries

TheRunningQueriestabgivesyouacompactoverviewofallrunningqueries.Byclickingtheredminusbutton,youcanaborttheexecutionofarunningquery.

SlowQueryHistory

Queries

TheSlowQueryHistorytabgivesyouacompactoverviewofallpastslowqueries.

Queries

GraphsTheGraphstabprovidesaviewerfacilityforgraphdatastoredinArangoDB.ItallowsbrowsingArangoDBgraphsstoredinthe_graphssystemcollectionoragraphconsistingofanarbitraryvertexandedgecollection.

Pleasenotethatthegraphviewerrequirescanvas(optional:webgl)supportinyourbrowser.EspeciallyInternetExplorerbrowsersolderthanversion9arelikelytonotsupportthis.

GraphViewer

Graphs

TopToolbarFunctions:

Loadfullgraph(Alsonodeswithoutconnectionswillbedrawn.Usefulduringgraphmodelingsetup)TakeagraphscreenshotStartfullscreenmodeOpengraphoptionsmenu

DefaultContextMenu(mouse-clickbackground):

AddanewnodeClosevisiblecontextmenu(s)

NodeContextMenu(mouse-clicknode):

DeletenodeEditnodeExpandnode(Showallboundedges)Drawedge(Connectwithanothernode)Setasstartnode(TheGraphwillrerenderstartingtheselectednodeandgivenoptions(graphoptionsmenu))

EdgeContextMenu(mouse-clickedge):

EditedgeDeleteedge

EdgeHighlighting(right-mouse-clicknode):

Highlightalledgesconnectedtothenode(right-clickatthebackgroundwillremovehighlighting)

Graphs

GraphViewerOptionsGraphOptionsMenu:

Startnode(string-validnodeidorspaceseperatedlistofid's):Heartofyourgraph.Renderingandtraversingwillstartfromhere.Emptyvaluemeans:arandomstartingpointwillbeused.Layout:Differentgraphlayoutingalgoritms.Nooverlap(optimal:biggraph),forcelayout(optimal:mediumgraph),fruchtermann(optimal:littletomediumgraph).Renderer:Canvasmodeallowsediting.WebGLcurrentlyoffersonlydisplaymode(alotfasterwithmuchnodes/edges).Searchdepth(number):Searchdepthwhichisstartingfromyourstartnode.Limit(number):Limitnodescount.Ifemptyorzero,nolimitisset.

NodesOptionsMenu:

Label(string):Nodeswillbelabeledbythisattribute.Ifnodeattributeisnotfound,nolabelwillbedisplayed.AddCollectionName:Thisappendsthecollectionnametothelabel,ifitexists.ColorByCollections:Shouldnodesbecolorizedbytheircollection?Ifenabled,nodecolorandnodecolorattributewillbeignored.Color:Defaultnodecolor.ColorAttribute(string):Ifanattributeisgiven,nodeswillthenbecolorizedbytheattribute.Thissettingignoresdefaultnodecolorifset.SizeByConnections:Shouldnodesbesizedbytheiredgescount?Ifenabled,nodesizingattributewillbeignored.SizingAttribute(number):Defaultnodesize.Numericvalue>0.

EdgesOptionsMenu:

Label(string):Edgeswillbelabeledbythisattribute.Ifedgeattributeisnotfound,nolabelwillbedisplayed.AddCollectionName:Thisappendsthecollectionnametothelabel,ifitexists.ColorByCollections:Shouldedgesbecolorizedbytheircollection?Ifenabled,edgecolorandedgecolorattributewillbeignored.Color:Defaultedgecolor.ColorAttribute(string):Ifanattributeisgiven,edgeswillthenbecolorizedbytheattribute.Thissettingignoresdefaultnodecolorifset.Type:Therendereroffersmultipletypesofrendering.Theyonlydifferintheirdisplaystyle,exceptforthetype'curved'.Thecurvedtypeallowstodisplaymorethanoneedgesbetweentwonodes.

Graphs

ServicesTheservicessectiondisplaysallinstalledfoxxapplications.Youcancreatenewservicesorgointoadetailedviewofachoosenservice.

CreateService

Therearefourdifferentpossibilitiestocreateanewservice:

1. Createserviceviazipfile2. Createserviceviagithubrepository3. CreateserviceviaofficialArangoDBstore4. Createablankservicefromscratch

Services

ServiceView

Thissectionoffersseveralinformationaboutaspecificservice.

Therearefourviewcategories:

1. Info:

Displaysname,shortdescription,license,version,mode(production,development)Offersabuttontogototheservicesinterface(ifavailable)

Services

2. Api:

DisplayAPIasSwaggerUIDisplayAPIasRAWJSON

3. Readme:

Displaystheservicesmanual(ifavailable)4. Settings:

DownloadserviceaszipfileRunservicetests(ifavailable)Runservicescripts(ifavailable)Configuredependencies(ifavailable)Changeserviceparameters(ifavailable)Changemode(production,development)ReplacetheserviceDeletetheservice

Services

ManagingUsersintheWebInterfaceArangoDBusersaregloballystoredinthe_systemdatabaseandcanonlybemanangedwhileloggedontothisdatabase.ThereyoucanfindtheUserssection:

General

SelectausertobringuptheGeneraltabwiththeusername,nameandactivestatus,aswellasoptionstodeletetheuserorchangethepassword.

Permissions

SelectauserandgotothePermissionstab.Youwillseealistofdatabasesandtheircorrespondingdatabaseaccesslevelforthatuser.

Pleasenotethatserveraccesslevelfollowsfromtheaccesslevelonthedatabase_system.Furthermore,thedefaultdatabaseaccesslevelforthisuserappearintheartificialrowwiththedatabasename*.

Belowthistableisanotheroneforthecollectioncategoryaccesslevels.Atfirst,itshowsthelistofdatabases,too.Ifyouclickonadatabase,thelistofcollectionsinthatdatabasewillbeopenandyoucanseethedefinedcollectionaccesslevelsforeachcollectionofthatdatabase(whichcanbeallunselectedwhichmeansthatnothingisexplicitlyset).Thedefaultaccesslevelsforthisuseranddatabaseappearintheartificialrowwiththecollectionname*.

AlsoseeManagingUsersaboutaccesslevels.

LogsThelogssectiondisplaysallavailablelogentries.Logentriesarefilterablebytheirlogleveltypes.

Functions:

Filterlogentriesbyloglevel(all,info,error,warning,debug)

Information:

LoglevelDateMessage

ArangoDBShellIntroductionTheArangoDBshell(arangosh)isacommand-linetoolthatcanbeusedforadministrationofArangoDB,includingrunningad-hocqueries.

ThearangoshbinaryisshippedwithArangoDB.ItoffersaJavaScriptshellenvironmentprovidingaccesstotheArangoDBserver.Arangoshcanbeinvokedlikethis:

unix>arangosh

BydefaultarangoshwilltrytoconnecttoanArangoDBserverrunningonserverlocalhostonport8529.Itwillusetheusernamerootandanemptypasswordbydefault.Additionallyitwillconnecttothedefaultdatabase(_system).Allthesedefaultscanbechangedusingthefollowingcommand-lineoptions:

--server.database:nameofthedatabasetoconnectto--server.endpoint:endpointtoconnectto--server.username:databaseusername--server.password:passwordtousewhenconnecting--server.authentication:whetherornottouseauthentication

Forexample,toconnecttoanArangoDBserveronIP192.168.173.13onport8530withtheuserfooandusingthedatabasetest,use:

unix>arangosh\

--server.endpointtcp://192.168.173.13:8530\

--server.usernamefoo\

--server.databasetest\

--server.authenticationtrue

arangoshwillthendisplayapasswordpromptandtrytoconnecttotheserverafterthepasswordwasentered.

Tochangethecurrentdatabaseaftertheconnectionhasbeenmade,youcanusethedb._useDatabase()commandinarangosh:

arangosh>db._createDatabase("myapp");

arangosh>db._useDatabase("myapp");

arangosh>db._useDatabase("_system");

arangosh>db._dropDatabase("myapp");

Togetalistofavailablecommands,arangoshprovidesahelp()function.Callingitwilldisplayhelpfulinformation.

arangoshalsoprovidesauto-completion.Additionalinformationonavailablecommandsandmethodsisthusprovidedbytypingthefirstfewlettersofavariableandthenpressingthetabkey.Itisrecommendtotrythiswithenteringdb.(withoutpressingreturn)andthenpressingtab.

Bytheway,arangoshprovidesthedbobjectbydefault,andthisobjectcanbeusedforswitchingtoadifferentdatabaseandmanagingcollectionsinsidethecurrentdatabase.

Foralistofavailablemethodsforthedbobject,type

arangosh>db._help();

showexecutionresultsyoucanpastemultiplelinesintoarangosh,giventhefirstlineendswithanopeningbrace:

arangosh>for(vari=0;i<10;i++){

ArangoDBShell

........>require("@arangodb").print("Helloworld"+i+"!\n");

........>}

showexecutionresultsToloadyourownJavaScriptcodeintothecurrentJavaScriptinterpretercontext,usetheloadcommand:

require("internal").load("/tmp/test.js")//<-Linux/MacOS

require("internal").load("c:\\tmp\\test.js")//<-Windows

Exitingarangoshcanbedoneusingthekeycombination<CTRL>+Dorbytypingquit<CR>

Escaping

InAQL,escapingisdonetraditionallywiththebackslashcharacter:\.Asseenabove,thisleadstodoublebackslasheswhenspecifyingWindowspaths.Arangoshrequiresanotherlevelofescaping,alsowiththebackslashcharacter.ItaddsuptofourbackslashesthatneedtobewritteninArangoshforasingleliteralbackslash(c:\tmp\test.js):

db._query('RETURN"c:\\\\tmp\\\\test.js"')

Youcanusebindvariablestomitigatethis:

varsomepath="c:\\tmp\\test.js"

db._query(aql`RETURN${somepath}`)

ArangoDBShell

ArangoDBShellOutputTheArangoDBshellwillprinttheoutputofthelastevaluatedexpressionbydefault:

arangosh>42*23

Inordertopreventprintingtheresultofthelastevaluatedexpression,theexpressionresultcanbecapturedinavariable,e.g.

arangosh>varcalculationResult=42*23

ThereisalsotheprintfunctiontoexplicitlyprintoutvaluesintheArangoDBshell:

arangosh>print({a:"123",b:[1,2,3],c:"test"});

showexecutionresultsBydefault,theArangoDBshellusesaprettyprinterwhenJSONdocumentsareprinted.Thisensuresdocumentsareprintedinahuman-readableway:

arangosh>db._create("five")

arangosh>for(i=0;i<5;i++)db.five.save({value:i})

arangosh>db.five.toArray()

showexecutionresultsWhilethepretty-printerproducesnicelookingresults,itwillneedalotofscreenspaceforeachdocument.Sometimesamoredenseoutputmightbebetter.Inthiscase,theprettyprintercanbeturnedoffusingthecommandstop_pretty_print().

Toturnonprettyprintingagain,usethestart_pretty_print()command.

ShellOutput

ArangoDBShellConfigurationarangoshwilllookforauser-definedstartupscriptnamed.arangosh.rcintheuser'shomedirectoryonstartup.Thehomedirectorywilllikelybe/home/<username>/onUnix/Linux,andisdeterminedonWindowsbypeekingintotheenvironmentvariables%HOMEDRIVE%and%HOMEPATH%.

Ifthefile.arangosh.rcispresentinthehomedirectory,arangoshwillexecutethecontentsofthisfileinsidetheglobalscope.

Youcanusethistodefineyourownextravariablesandfunctionsthatyouneedoften.Forexample,youcouldputthefollowingintothe.arangosh.rcfileinyourhomedirectory:

//"var"keywordavoidedintentionally...

//otherwise"timed"wouldnotsurvivethescopeofthisscript

global.timed=function(cb){

console.time("callback");

console.timeEnd("callback");

Thiswillmakeafunctionnamedtimedavailableinarangoshintheglobalscope.

Youcannowstartarangoshandinvokethefunctionlikethis:

timed(function(){

for(vari=0;i<1000;++i){

db.test.save({value:i});

Pleasekeepinmindthat,ifpresent,the.arangosh.rcfileneedstocontainvalidJavaScriptcode.Ifyouwantanyvariablesintheglobalscopetosurviveyouneedtoomitthevarkeywordforthem.Otherwisethevariableswillonlybevisibleinsidethescriptitself,butnotoutside.

Configuration

DetailsabouttheArangoDBShellAftertheserverhasbeenstarted,youcanusetheArangoDBshell(arangosh)toadministratetheserver.Withoutanyarguments,theArangoDBshellwilltrytocontacttheserveronport8529onthelocalhost.FormoreinformationseetheArangoDBShelldocumentation.Youmightneedtosetadditionaloptions(endpoint,usernameandpassword)whenconnecting:

unix>./arangosh--server.endpointtcp://127.0.0.1:8529--server.usernameroot

TheshellwillprintitsownversionnumberandifsuccessfullyconnectedtoaservertheversionnumberoftheArangoDBserver.

Command-LineOptionsUse--helptogetalistofcommand-lineoptions:

unix>./arangosh--help

STANDARDoptions:

--audit-log<string>auditlogfiletosavecommandsandresultsto

--configuration<string>readconfigurationfile

--helphelpmessage

--max-upload-size<uint64>maximumsizeofimportchunks(inbytes)(default:500000)

--no-auto-completedisableautocompletion

--no-colorsdeactivatecolorsupport

--pager<string>outputpager(default:"less-X-R-F-L")

--pretty-printprettyprintvalues

--quietnobanner

--temp.path<string>pathfortemporaryfiles(default:"/tmp/arangodb")

--use-pagerusepager

JAVASCRIPToptions:

--javascript.check<string>syntaxcheckcodeJavaScriptcodefromfile

--javascript.execute<string>executeJavaScriptcodefromfile

--javascript.execute-string<string>executeJavaScriptcodefromstring

--javascript.startup-directory<string>startuppathscontainingtheJavaScriptfiles

--javascript.unit-tests<string>donotstartasshell,rununittestsinstead

--jslint<string>donotstartasshell,runjslintinstead

LOGGINGoptions:

--log.level<string>loglevel(default:"info")

CLIENToptions:

--server.connect-timeout<double>connecttimeoutinseconds(default:3)

--server.authentication<bool>whetherornottouseauthentication(default:true)

--server.endpoint<string>endpointtoconnectto,use'none'tostartwithoutaserver(default:"tcp://127.0

.0.1:8529")

--server.password<string>passwordtousewhenconnecting(leaveemptyforprompt)

--server.request-timeout<double>requesttimeoutinseconds(default:300)

--server.username<string>usernametousewhenconnecting(default:"root")

DatabaseWrappersThedbobjectisavailableinarangoshaswellasonarangodi.e.ifyou'reusingFoxx.Whileitsinterfaceispersistantbetweenthearangoshandthearangodimplementations,itsunderpinningisnot.ThearangodimplementationareJavaScriptwrappersaroundArangoDB'snativeC++implementation,whereasthearangoshimplementationwrapsHTTPaccessestoArangoDB'sRESTfullAPI.

Sowhilethiscodemayproducesimilarresultswhenexecutedinarangoshandarangod,thecpuusageandtimerequiredwillbereallydifferent:

for(i=0;i<100000;i++){

db.test.save({name:{first:"Jan"},count:i});

Sincethearangoshversionwillbedoingaround100kHTTPrequests,andthearangodversionwilldirectlywritetothedatabase.

Details

Using arangoshviaunixshebangmechanisms

Inunixoperatingsystemsyoucanstartscriptsbyspecifyingtheinterpreterinthefirstlineofthescript.Thisiscommonlycalledshebangorhashbang.Youcanalsodothatwitharangosh,i.e.create~/test.js:

#!/usr/bin/arangosh--javascript.execute

require("internal").print("helloworld")

db._query("FORxINtestRETURNx").toArray()

Notethatthefirstlinehastoendwithablankinordertomakeitwork.MarkitexecutabletotheOS:

#>chmoda+x~/test.js

andfinalytryitout:

#>~/test.js

Details

ArangoimportThismanualdescribestheArangoDBimporterarangoimport,whichcanbeusedforbulkimports.

ThemostconvenientmethodtoimportalotofdataintoArangoDBistousethearangoimportcommand-linetool.Itallowsyoutoimportdatarecordsfromafileintoanexistingdatabasecollection.

Itispossibletoimportdocumentkeyswiththedocumentsusingthe_keyattribute.Whenimportingintoanedgecollection,itismandatorythatallimporteddocumentshavethe_fromand_toattributes,andthattheycontainvalidreferences.

Let'sassumeforthefollowingexamplesyouwanttoimportuserdataintoanexistingcollectionnamed"users"ontheserver.

ImportingDataintoanArangoDBDatabase

ImportingJSON-encodedData

Let'sfurtherassumetheimportathandisencodedinJSON.We'llbeusingtheseexampleuserrecordstoimport:

{"name":{"first":"John","last":"Connor"},"active":true,"age":25,"likes":["swimming"]}

{"name":{"first":"Jim","last":"O'Brady"},"age":19,"likes":["hiking","singing"]}

{"name":{"first":"Lisa","last":"Jones"},"dob":"1981-04-09","likes":["running"]}

Toimporttheserecords,allyouneedtodoistoputthemintoafile(withonelineforeachrecordtoimport)andrunthefollowingcommand:

>arangoimport--file"data.json"--typejsonl--collection"users"

Thiswilltransferthedatatotheserver,importtherecords,andprintastatussummary.Toshowtheintermediateprogressduringtheimportprocess,theoption--progresscanbeadded.Thisoptionwillshowthepercentageoftheinputfilethathasbeensenttotheserver.Thiswillonlybeusefulforbigimportfiles.

>arangoimport--file"data.json"--typejson--collectionusers--progresstrue

Itisalsopossibletousetheoutputofanothercommandasaninputforarangoimport.Forexample,thefollowingshellcommandcanbeusedtopipedatafromthecatprocesstoarangoimport:

>catdata.json|arangoimport--file---typejson--collectionusers

Notethatyouhavetouse--file-ifyouwanttouseanothercommandasinputforarangoimport.Noprogresscanbereportedforsuchimportsasthesizeoftheinputwillbeunknowntoarangoimport.

Bydefault,theendpointtcp://127.0.0.1:8529willbeused.Ifyouwanttospecifyadifferentendpoint,youcanusethe--server.endpointoption.Youprobablywanttospecifyadatabaseuserandpasswordaswell.Youcandosobyusingtheoptions--server.usernameand--server.password.Ifyoudonotspecifyapassword,youwillbepromptedforone.

>arangoimport--server.endpointtcp://127.0.0.1:8529--server.usernameroot--file"data.json"--typejson--collection"users

Notethatthecollection(usersinthiscase)mustalreadyexistortheimportwillfail.Ifyouwanttocreateanewcollectionwiththeimportdata,youneedtospecifythe--create-collectionoption.Notethatbydefaultitwillcreateadocumentcollectionandnoedgecollection.

>arangoimport--file"data.json"--typejson--collection"users"--create-collectiontrue

Tocreateanedgecollectioninstead,usethe--create-collection-typeoptionandsetittoedge:

Arangoimport

>arangoimport--file"data.json"--collection"myedges"--create-collectiontrue--create-collection-typeedge

Whenimportingdataintoanexistingcollectionitisoftenconvenienttofirstremovealldatafromthecollectionandthenstarttheimport.Thiscanbeachievedbypassingthe--overwriteparametertoarangoimport.Ifitissettotrue,anyexistingdatainthecollectionwillberemovedpriortotheimport.Notethatanyexistingindexdefinitionsforthecollectionwillbepreservedevenif--overwriteissettotrue.

>arangoimport--file"data.json"--typejson--collection"users"--overwritetrue

AstheimportfilealreadycontainsthedatainJSONformat,attributenamesanddatatypesarefullypreserved.Ascanbeseenintheexampledata,thereisnoneedforalldatarecordstohavethesameattributenamesortypes.Recordscanbeinhomogeneous.

Pleasenotethatbydefault,arangoimportwillimportdataintothespecifiedcollectioninthedefaultdatabase(_system).Tospecifyadifferentdatabase,usethe--server.databaseoptionwheninvokingarangoimport.Ifyouwanttoimportintoanonexistentdatabaseyouneedtopass--create-databasetrue.Note--create-databasedefaultstofalse

Thetoolalsosupportsparallelimports,withmultiplethreads.Usingmultiplethreadsmayprovideaspeedup,especiallywhenusingtheRocksDBstorageengine.Tospecifythenumberofparallelthreadsusethe--threadsoption:

>arangoimport--threads4--file"data.json"--typejson--collection"users"

Notethatusingmultiplethreadsmayleadtoanon-sequentialimportoftheinputdata.Datathatappearslaterintheinputfilemaybeimportedearlierthandatathatappearsearlierintheinputfile.Thisisnormallynotaproblembutmaycauseissueswhenwhentherearedatadependenciesorduplicatesintheimportdata.Inthiscase,thenumberofthreadsshouldbesetto1.

JSONinputfileformats

Note:arangoimportsupportstwoformatswhenimportingJSONdatafromafile.Thefirstformatthatwealsousedaboveiscommonlyknownasjsonl).However,incontrasttotheJSONLspecificationitrequirestheinputfiletocontainonecompleteJSONdocumentineachline,e.g.

{"_key":"one","value":1}

{"_key":"two","value":2}

{"_key":"foo","value":"bar"}

SoonecouldarguethatthisisonlyasubsetofJSONL.

Theaboveformatcanbeimportedsequentiallybyarangoimport.Itwillreaddatafromtheinputfileinchunksandsenditinbatchestotheserver.Eachbatchwillbeaboutasbigasspecifiedinthecommand-lineparameter--batch-size.

AnalternativeistoputonebigJSONdocumentintotheinputfilelikethis:

{"_key":"one","value":1},

{"_key":"two","value":2},

{"_key":"foo","value":"bar"},

Thisformatallowslinebreakswithintheinputfileasrequired.Thedownsideisthatthewholeinputfilewillneedtobereadbyarangoimportbeforeitcansendthefirstbatch.Thismightbeaproblemiftheinputfileisbig.Bydefault,arangoimportwillallowimportingsuchfilesuptoasizeofabout16MB.

Ifyouwanttoallowyourarangoimportinstancetousemorememory,youmaywanttoincreasethemaximumfilesizebyspecifyingthecommand-lineoption--batch-size.Forexample,tosetthebatchsizeto32MB,usethefollowingcommand:

>arangoimport--file"data.json"--typejson--collection"users"--batch-size33554432

Pleasealsonotethatyoumayneedtoincreasethevalueof--batch-sizeifasingledocumentinsidetheinputfileisbiggerthanthevalueof--batch-size.

Arangoimport

ImportingCSVData

arangoimportalsooffersthepossibilitytoimportdatafromCSVfiles.ThiscomeshandywhenthedataathandisinCSVformatalreadyandyoudon'twanttospendtimeconvertingthemtoJSONfortheimport.

ToimportdatafromaCSVfile,makesureyourfilecontainstheattributenamesinthefirstrow.Allthefollowinglinesinthefilewillbeinterpretedasdatarecordsandwillbeimported.

TheCSVimportrequiresthedatatohaveahomogeneousstructure.Allrecordsmusthaveexactlythesameamountofcolumnsasthereareheaders.Bydefault,lineswithadifferentnumberofvalueswillnotbeimportedandtherewillbewarningsforthem.Tostillimportlineswithlessvaluesthanintheheader,thereisthe--ignore-missingoption.Ifsettotrue,linesthathaveadifferentamountoffieldswillbeimported.Inthiscaseonlythoseattributeswillbepopulatedforwhichtherearevalues.Attributesforwhichtherearenovaluespresentwillsilentlybediscarded.

Example:

"first","last","age","active","dob"

"John","Connor",25,true

"Jim","O'Brady"

With--ignore-missingthiswillproducethefollowingdocuments:

{"first":"John","last":"Connor","active":true,"age":25}

{"first":"Jim","last":"O'Brady"}

Thecellvaluescanhavedifferentdatatypesthough.Ifacelldoesnothaveanyvalue,itcanbeleftemptyinthefile.Thesevalueswillnotbeimportedsotheattributeswillnot"bethere"indocumentcreated.Valuesenclosedinquoteswillbeimportedasstrings,sotoimportnumericvalues,booleanvaluesorthenullvalue,don'tenclosethevalueinquotesinyourfile.

We'llbeusingthefollowingimportfortheCSVimport:

"first","last","age","active","dob"

"John","Connor",25,true,

"Jim","O'Brady",19,,

"Lisa","Jones",,,"1981-04-09"

Hans,dosSantos,0123,,

Wayne,Brewer,,false,

Thecommandlinetoexecutetheimportis:

>arangoimport--file"data.csv"--typecsv--collection"users"

Theabovedatawillbeimportedinto5documentswhichwilllookasfollows:

{"first":"John","last":"Connor","active":true,"age":25}

{"first":"Jim","last":"O'Brady","age":19}

{"first":"Lisa","last":"Jones","dob":"1981-04-09"}

{"first":"Hans","last":"dosSantos","age":123}

{"first":"Wayne","last":"Brewer","active":false}

Ascanbeseen,valuesleftcompletelyemptyintheinputfilewillbetreatedasabsent.Numericvaluesnotenclosedinquoteswillbetreatedasnumbers.Notethatleadingzerosinnumericvalueswillberemoved.Toimportnumberswithleadingzeros,pleaseusestrings.Theliteralstrueandfalsewillbetreatedasbooleansiftheyarenotenclosedinquotes.Othervaluesnotenclosedinquoteswillbetreatedasstrings.Anyvaluesenclosedinquoteswillbetreatedasstrings,too.

Stringvaluescontainingthequotecharacterortheseparatormustbeenclosedwithquotecharacters.Withinastring,thequotecharacteritselfmustbeescapedwithanotherquotecharacter(orwithabackslashifthe--backslash-escapeoptionisused).

Notethatthequoteandseparatorcharacterscanbeadjustedviathe--quoteand--separatorargumentswheninvokingarangoimport.Thequotecharacterdefaultstothedoublequote(").Tousealiteralquoteinastring,youcanusetwoquotecharacters.Tousebackslashforescapingquotecharacters,pleasesettheoption--backslash-escapetotrue.

Arangoimport

TheimportersupportsWindows(CRLF)andUnix(LF)linebreaks.Linebreaksmightalsooccurinsidevaluesthatareenclosedwiththequotecharacter.

Here'sanexampleforusingliteralquotesandnewlinesinsidevalues:

"name","password"

"Foo","r4ndom""123!"

"Bar","wow!

thisisa

multinepassword!"

"Bartholomew""Bart""Simpson","Milhouse"

Extrawhitespaceattheendofeachlinewillbeignored.Whitespaceatthestartoflinesorbetweenfieldvalueswillnotbeignored,sopleasemakesurethatthereisnoextrawhitespaceinfrontofvaluesorbetweenthem.

ImportingTSVData

Youmayalsoimporttab-separatedvalues(TSV)fromafile.Thisformatisverysimple:everylineinthefilerepresentsadatarecord.Thereisnoquotingorescaping.Thatalsomeansthattheseparatorcharacter(whichdefaultstothetabstopsymbol)mustnotbeusedanywhereintheactualdata.

AswithCSV,thefirstlineintheTSVfilemustcontaintheattributenames,andalllinesmusthaveanidenticalnumberofvalues.

Ifadifferentseparatorcharacterorstringshouldbeused,itcanbespecifiedwiththe--separatorargument.

AnexamplecommandlinetoexecutetheTSVimportis:

>arangoimport--file"data.tsv"--typetsv--collection"users"

AttributeNameTranslation

FortheCSVandTSVinputformats,attributenamescanbetranslatedautomatically.ThisisusefulincasetheimportfilehasdifferentattributenamesthanthosethatshouldbeusedinArangoDB.

Acommonusecaseistorenamean"id"columnfromtheinputfileinto"_key"asitisexpectedbyArangoDB.Todothis,specifythefollowingtranslationwheninvokingarangoimport:

>arangoimport--file"data.csv"--typecsv--translate"id=_key"

Othercommoncasesaretorenamecolumnsintheinputfileto_fromand_to:

>arangoimport--file"data.csv"--typecsv--translate"from=_from"--translate"to=_to"

Thetranslateoptioncanbespecifiedmultipletypes.Thesourceattributenameandthetargetattributemustbeseparatedwitha=.

IgnoringAttributes

FortheCSVandTSVinputformats,certainattributenamescanbeignoredonimports.InanArangoDBclustertherearecaseswherethiscancomeinhandy,whenyourdocumentsalreadycontaina_keyattributeandyourcollectionhasashardingattributeotherthan_key:Intheclusterthisconfigurationisnotsupported,becauseArangoDBneedstoguaranteetheuniquenessofthe_keyattributeinallshardsofthecollection.

>arangoimport--file"data.csv"--typecsv--remove-attribute"_key"

Thesamethingwouldapplyifyourdatacontainsan_idattribute:

>arangoimport--file"data.csv"--typecsv--remove-attribute"_id"

ImportingintoanEdgeCollection

Arangoimport

arangoimportcanalsobeusedtoimportdataintoanexistingedgecollection.Theimportdatamust,foreachedgetoimport,containatleastthe_fromand_toattributes.Theseindicatewhichothertwodocumentstheedgeshouldconnect.Itisnecessarythattheseattributesaresetforallrecords,andpointtovaliddocumentidsinexistingcollections.

Examples

{"_from":"users/1234","_to":"users/4321","desc":"1234isconnectedto4321"}

Note:Theedgecollectionmustalreadyexistwhentheimportisstarted.Usingthe--create-collectionflagwillnotworkbecausearangoimportwillalwaystrytocreatearegulardocumentcollectionifthetargetcollectiondoesnotexist.

Updatingexistingdocuments

Bydefault,arangoimportwilltrytoinsertalldocumentsfromtheimportfileintothespecifiedcollection.Incasetheimportfilecontainsdocumentsthatarealreadypresentinthetargetcollection(matchingisdoneviathe_keyattributes),thenadefaultarangoimportrunwillnotimportthesedocumentsandcomplainaboutuniquekeyconstraintviolations.

However,arangoimportcanbeusedtoupdateorreplaceexistingdocumentsincasetheyalreadyexistinthetargetcollection.Itprovidesthecommand-lineoption--on-duplicatetocontrolthebehaviorincaseadocumentisalreadypresentinthedatabase.

Thedefaultvalueof--on-duplicateiserror.Thismeansthatwhentheimportfilecontainsadocumentthatispresentinthetargetcollectionalready,thentryingtore-insertadocumentwiththesame_keyvalueisconsideredanerror,andthedocumentinthedatabasewillnotbemodified.

Otherpossiblevaluesfor--on-duplicateare:

update:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbeupdatedbyarangoimport.updatewillperformapartialupdateoftheexistingdocument,modifyingonlytheattributesthatarepresentintheimportfileandleavingallotherattributesuntouched.

Thevaluesofsystemattributes_id,_key,_rev,_fromand_tocannotbeupdatedorreplacedinexistingdocuments.

replace:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbereplacebyarangoimport.replacewillreplacetheexistingdocumententirely,resultinginadocumentwithonlytheattributesspecifiedintheimportfile.

Thevaluesofsystemattributes_id,_key,_rev,_fromand_tocannotbeupdatedorreplacedinexistingdocuments.

ignore:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbeignoredandnotmodifiedinthetargetcollection.

When--on-duplicateissettoeitherupdateorreplace,arangoimportwillreturnthenumberofdocumentsupdated/replacedintheupdatedreturnvalue.Whensettoanothervalue,thevalueofupdatedwillalwaysbezero.When--on-duplicateissettoignore,arangoimportwillreturnthenumberofignoreddocumentsintheignoredreturnvalue.Whensettoanothervalue,ignoredwillalwaysbezero.

Itispossibletoperformacombinationofinsertsandupdates/replaceswithasinglearangoimportrun.When--on-duplicateissettoupdateorreplace,alldocumentspresentintheimportfilewillbeinsertedintothetargetcollectionprovidedtheyarevalidanddonotalreadyexistwiththespecified_key.Documentsthatarealreadypresentinthetargetcollection(identifiedby_keyattribute)willinsteadbeupdated/replaced.

Arangoimportresultoutput

Anarangoimportimportrunwillprintoutthefinalresultsonthecommandline.Itwillshowthe

numberofdocumentscreated(created)numberofdocumentsupdated/replaced(updated/replaced,onlynon-zeroif--on-duplicatewassettoupdateorreplace,seebelow)numberofwarningsorerrorsthatoccurredontheserverside(warnings/errors)numberofignoreddocuments(onlynon-zeroif--on-duplicatewassettoignore).

Example

created:2

warnings/errors:0

updated/replaced:0

ignored:0

Arangoimport

ForCSVandTSVimports,thetotalnumberofinputfilelinesreadwillalsobeprinted(linesread).

arangoimportwillalsoprintoutdetailsaboutwarningsanderrorsthathappenedontheserver-side(ifany).

AttributeNamingandSpecialAttributes

AttributeswhosenamesstartwithanunderscorearetreatedinaspecialwaybyArangoDB:

theoptional_keyattributecontainsthedocument'skey.Ifspecified,thevaluemustbeformallyvalid(e.g.mustbeastringandconformtothenamingconventions).Additionally,thekeyvaluemustbeuniquewithinthecollectiontheimportisrunfor._from:whenimportingintoanedgecollection,thisattributecontainstheidofoneofthedocumentsconnectedbytheedge.Thevalueof_frommustbeasyntacticallyvaliddocumentidandthereferredcollectionmustexist._to:whenimportingintoanedgecollection,thisattributecontainstheidoftheotherdocumentconnectedbytheedge.Thevalueof_tomustbeasyntacticallyvaliddocumentidandthereferredcollectionmustexist._rev:thisattributecontainstherevisionnumberofadocument.However,therevisionnumbersaremanagedbyArangoDBandcannotbespecifiedonimport.Thusanyvalueinthisattributeisignoredonimport.

Ifyouimportvaluesinto_key,youshouldmakesuretheyarevalidandunique.

Whenimportingdataintoanedgecollection,youshouldmakesurethatallimportdocumentscan_fromand_toandthattheirvaluespointtoexistingdocuments.

Toavoidspecifyingcompletedocumentids(consistingofcollectionnamesanddocumentkeys)for_fromand_tovalues,therearetheoptions--from-collection-prefixand--to-collection-prefix.Ifspecified,thesevalueswillbeautomaticallyprependedtoeachvaluein_from(or_toresp.).Thisallowsspecifyingonlydocumentkeysinside_fromand/or_to.

Example

>arangoimport--from-collection-prefixusers--to-collection-prefixproducts...

Importingthefollowingdocumentwillthencreateanedgebetweenusers/1234andproducts/4321:

{"_from":"1234","_to":"4321","desc":"users/1234isconnectedtoproducts/4321"}

Arangoimport

DumpingDatafromanArangoDBdatabaseTodumpdatafromanArangoDBserverinstance,youwillneedtoinvokearangodump.Dumpscanbere-importedwitharangorestore.arangodumpcanbeinvokedbyexecutingthefollowingcommand:

unix>arangodump--output-directory"dump"

ThiswillconnecttoanArangoDBserveranddumpallnon-systemcollectionsfromthedefaultdatabase(_system)intoanoutputdirectorynameddump.Invokingarangodumpwillfailiftheoutputdirectoryalreadyexists.Thisisanintentionalsecuritymeasuretopreventyoufromaccidentallyoverwritingalreadydumpeddata.Ifyouarepositivethatyouwanttooverwritedataintheoutputdirectory,youcanusetheparameter--overwritetruetoconfirmthis:

unix>arangodump--output-directory"dump"--overwritetrue

arangodumpwillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabaseoradifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:

--server.database:nameofthedatabasetoconnectto--server.endpoint:endpointtoconnectto--server.username:username--server.password:passwordtouse(omitthisandyou'llbepromptedforthepassword)--server.authentication:whetherornottouseauthentication

Here'sanexampleofdumpingdatafromanon-standardendpoint,usingadedicateddatabasename:

unix>arangodump--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--output-directory

"dump"

Whenfinished,arangodumpwillprintoutasummarylinewithsomeaggregatestatisticsaboutwhatitdid,e.g.:

Processed43collection(s),wrote408173500byte(s)intodatafiles,sent88batch(es)

Bydefault,arangodumpwilldumpbothstructuralinformationanddocumentsfromallnon-systemcollections.Toadjustthis,therearethefollowingcommand-linearguments:

--dump-data:settotruetoincludedocumentsinthedump.Settofalsetoexcludedocuments.Thedefaultvalueistrue.--include-system-collections:whetherornottoincludesystemcollectionsinthedump.Thedefaultvalueisfalse.

Forexample,toonlydumpstructuralinformationofallcollections(includingsystemcollections),use:

unix>arangodump--dump-datafalse--include-system-collectionstrue--output-directory"dump"

Torestrictthedumptojustspecificcollections,thereisisthe--collectionoption.Itcanbespecifiedmultipletimesifrequired:

unix>arangodump--collectionmyusers--collectionmyvalues--output-directory"dump"

Structuralinformationforacollectionwillbesavedinfileswithnamepattern.structure.json.EachstructurefilewillcontainsaJSONobjectwiththeseattributes:

parameters:containsthecollectionpropertiesindexes:containsthecollectionindexes

Documentdataforacollectionwillbesavedinfileswithnamepattern.data.json.Eachlineinadatafileisadocumentinsertion/updateordeletionmarker,alongsidewithsomemetadata.

StartingwithVersion2.1ofArangoDB,thearangodumptoolalsosupportssharding.Simplypointittooneofthecoordinatorsanditwillbehaveexactlyasdescribedabove,workingonshardedcollectionsinthecluster.

Arangodump

However,asopposedtothesingleinstancesituation,thisoperationdoesnotguaranteetodumpaconsistentsnapshotifwriteoperationshappenduringthedumpoperation.Itisthereforerecommendednottoperformanydata-modifcationoperationsontheclusterwhilstarangodumpisrunning.

Asabove,theoutputwillbeonestructuredescriptionfileandonedatafilepershardedcollection.Notethatthedatainthedatafileissortedfirstbyshardsandwithineachshardbyascendingtimestamp.Thestructuralinformationofthecollectioncontainsthenumberofshardsandtheshardkeys.

NotethattheversionofthearangodumpclienttoolneedstomatchtheversionoftheArangoDBserveritconnectsto.

Advancedclusteroptions

Startingwithversion3.1.17,collectionsmaybecreatedwithsharddistributionidenticaltoanexistingprototypicalcollection;i.e.shardsaredistributedintheverysamepatternasintheprototypecollection.Suchcollectionscannotbedumpedwithoutthereferencecollectionorarangodumpwithyieldanerror.

unix>arangodump--collectionclonedCollection--output-directory"dump"

ERRORCollectionclonedCollection'ssharddistributionisbasedonathatofcollectionprototypeCollection,whichisnotdumpe

dalong.Youmaydumpthecollectionregardlessofthemissingprototypecollectionbyusingthe--ignore-distribute-shards-lik

e-errorsparameter.

Therearetwowaystoapproachthatproblem:Solveit,i.e.dumptheprototypecollectionalong:

unix>arangodump--collectionclonedCollection--collectionprototypeCollection--output-directory"dump"

Oroverridethatbehaviourtobeabletodumpthecollectionindividually.

unix>arangodump--collectionBclonedCollection--output-directory"dump"--ignore-distribute-shards-like-errors

Nothatinconsequence,restoringsuchacollectionwithoutitsprototypeisaffected.arangorestore

Encryption

IntheArangoDBEnterpriseEditiontherearetheadditionalparameters:

Encryptionkeystoredinfile

--encryption.keyfilepath-of-keyfile

Thefilepath-to-keyfilemustcontaintheencryptionkey.Thisfilemustbesecured,sothatonlyarangodcanaccessit.Youshouldalsoensurethatincasesome-onestealsthehardware,hewillnotbeabletoreadthefile.Forexample,byencryption/mytmpfsorcreatingain-memoryfile-systemunder/mytmpfs.

Encryptionkeygeneratedbyaprogram

--encryption.key-generatorpath-to-my-generator

Theprogrampath-to-my-generatormustoutputtheencryptiononstandardoutputandexit.

Creatingkeys

Theencryptionkeyfilemustcontain32bytesofrandomdata.

Youcancreateitwithacommandlinethis.

ddif=/dev/randombs=1count=32of=yourSecretKeyFile

Arangodump

Forsecurity,itisbesttocreatethesekeysoffline(awayfromyourdatabaseservers)anddirectlystoretheminyousecretmanagementtool.

Arangodump

ArangorestoreToreloaddatafromadumppreviouslycreatedwitharangodump,ArangoDBprovidesthearangorestoretool.

Pleasenotethatarangorestoremustnotbeusedtocreateseveralsimilardatabaseinstancesinoneinstallation.

Thismeansifyouhaveanarangodumpoutputofdatabasea,andyoucreateaseconddatabasebonthesameinstanceofArangoDB,andrestorethedumpofaintob-dataintegritycannotbeguaranteed.

ReloadingDataintoanArangoDBdatabase

Invokingarangorestore

arangorestorecanbeinvokedfromthecommand-lineasfollows:

unix>arangorestore--input-directory"dump"

ThiswillconnecttoanArangoDBserverandreloadstructuralinformationanddocumentsfoundintheinputdirectorydump.Pleasenotethattheinputdirectorymusthavebeencreatedbyrunningarangodumpbefore.

arangorestorewillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabaseoradifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:

Sinceversion2.6arangorestoreprovidestheoption--create-database.Settingthisoptiontotruewillcreatethetargetdatabaseifitdoesnotexist.Whencreatingthetargetdatabase,theusernameandpasswordspassedtoarangorestore(inoptions--server.usernameand--server.password)willbeusedtocreateaninitialuserforthenewdatabase.

Theoption--force-same-databaseallowsrestrictingarangorestoreoperationstoadatabasewiththesamenameasinthesourcedump's"dump.json"file.Itcanthusbeusedtopreventrestoringdataintoa"wrong"databasebyaccident.

Forexample,ifadumpwastakenfromdatabasea,andtherestoreisattemptedintodatabaseb,thenwiththe--force-same-databaseoptionsettotrue,arangorestorewillabortinstantly.

The--force-same-databaseoptionissettofalsebydefaulttoensurebackwards-compatibility.

Here'sanexampleofreloadingdatatoanon-standardendpoint,usingadedicateddatabasename:

unix>arangorestore--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--input-directo

ry"dump"

Tocreatethetargetdatabasewherestoring,useacommandlikethis:

unix>arangorestore--server.usernamebackup--server.databasenewdb--create-databasetrue--input-directory"dump"

arangorestorewillprintoutitsprogresswhilerunning,andwillendwithalineshowingsomeaggregatestatistics:

Processed2collection(s),read2256byte(s)fromdatafiles,sent2batch(es)

Bydefault,arangorestorewillre-createallnon-systemcollectionsfoundintheinputdirectoryandloaddataintothem.Ifthetargetdatabasealreadycontainscollectionswhicharealsopresentintheinputdirectory,theexistingcollectionsinthedatabasewillbedroppedandre-createdwiththedatafoundintheinputdirectory.

Arangorestore

Thefollowingparametersareavailabletoadjustthisbehavior:

--create-collection:settotruetocreatecollectionsinthetargetdatabase.Ifthetargetdatabasealreadycontainsacollectionwiththesamename,itwillbedroppedfirstandthenre-createdwiththepropertiesfoundintheinputdirectory.Settofalsetokeepexistingcollectionsinthetargetdatabase.Ifsettofalseandarangorestoreencountersacollectionthatispresentintheinputdirectorybutnotinthetargetdatabase,itwillabort.Thedefaultvalueistrue.--import-data:settotruetoloaddocumentdataintothecollectionsinthetargetdatabase.Settofalsetonotloadanydocumentdata.Thedefaultvalueistrue.--include-system-collections:whetherornottoincludesystemcollectionswhenre-creatingcollectionsorreloadingdata.Thedefaultvalueisfalse.

Forexample,to(re-)createallnon-systemcollectionsandloaddocumentdataintothem,use:

unix>arangorestore--create-collectiontrue--import-datatrue--input-directory"dump"

Thiswilldroppotentiallyexistingcollectionsinthetargetdatabasethatarealsopresentintheinputdirectory.

Toincludesystemcollectionstoo,use--include-system-collectionstrue:

unix>arangorestore--create-collectiontrue--import-datatrue--include-system-collectionstrue--input-directory"dump"

To(re-)createallnon-systemcollectionswithoutloadingdocumentdata,use:

unix>arangorestore--create-collectiontrue--import-datafalse--input-directory"dump"

Thiswillalsodropexistingcollectionsinthetargetdatabasethatarealsopresentintheinputdirectory.

Tojustloaddocumentdataintoallnon-systemcollections,use:

unix>arangorestore--create-collectionfalse--import-datatrue--input-directory"dump"

Torestrictreloadingtojustspecificcollections,thereisisthe--collectionoption.Itcanbespecifiedmultipletimesifrequired:

unix>arangorestore--collectionmyusers--collectionmyvalues--input-directory"dump"

Collectionswillbeprocessedbyinalphabeticalorderbyarangorestore,withalldocumentcollectionsbeingprocessedbeforealledgecollections.Thisistoensurethatreloadingdataintoedgecollectionswillhavethedocumentcollectionslinkedinedges(_fromand_toattributes)loaded.

Encryption

Seearangodumpfordetails.

RestoringRevisionIdsandCollectionIds

arangorestorewillreloaddocumentandedgesdatawiththeexactsame_key,_fromand_tovaluesfoundintheinputdirectory.However,whenloadingdocumentdata,itwillassignitsownvaluesforthe_revattributeofthereloadeddocuments.Thoughthisdifferenceisintentional(normally,everyservershouldcreateitsown_revvalues)theremightbesituationswhenitisrequiredtore-usetheexactsame_revvaluesforthereloadeddata.Thiscanbeachievedbysettingthe--recycle-idsparametertotrue:

unix>arangorestore--collectionmyusers--collectionmyvalues--input-directory"dump"

Notethatsetting--recycle-idstotruewillalsocausecollectionstobe(re-)createdinthetargetdatabasewiththeexactsamecollectionidasintheinputdirectory.Anypotentiallyexistingcollectioninthetargetdatabasewiththesamecollectionidwillthenbedropped.

ReloadingDataintoadifferentCollection

Arangorestore

Withsomecreativityyoucanusearangodumpandarangorestoretotransferdatafromonecollectionintoanother(eitheronthesameserverornot).Forexample,tocopydatafromacollectionmyvaluesindatabasemydbintoacollectionmycopyvaluesindatabasemycopy,youcanstartwiththefollowingcommand:

unix>arangodump--collectionmyvalues--server.databasemydb--output-directory"dump"

Thiswillcreatetwofiles,myvalues.structure.jsonandmyvalues.data.json,intheoutputdirectory.Toloaddatafromthedatafileintoanexistingcollectionmycopyvaluesindatabasemycopy,renamethefilestomycopyvalues.structure.jsonandmycopyvalues.data.json.Afterthat,runthefollowingcommand:

unix>arangorestore--collectionmycopyvalues--server.databasemycopy--input-directory"dump"

Usingarangorestorewithsharding

AsofVersion2.1thearangorestoretoolsupportssharding.Simplypointittooneofthecoordinatorsinyourclusteranditwillworkasusualbutonshardedcollectionsinthecluster.

Ifarangorestoreisaskedtodropandre-createacollection,itwillusethesamenumberofshardsandthesameshardkeysaswhenthecollectionwasdumped.Thedistributionoftheshardstotheserverswillalsobethesameasatthetimeofthedump.ThismeansinparticularthatDBserverswiththesameIDsasbeforemustbepresentintheclusterattimeoftherestore.

Ifacollectionwasdumpedfromasingleinstance,onecanmanuallyaddthestructuraldescriptionfortheshardkeysandthenumberanddistributionoftheshardsandthentherestoreintoaclusterwillwork.

IfyourestoreacollectionthatwasdumpedfromaclusterintoasingleArangoDBinstance,thenumberofshardsandtheshardkeyswillsilentlybeignored.

Notethatinacluster,everynewlycreatedcollectionwillhaveanewID,itisnotpossibletoreusetheIDfromtheoriginallydumpedcollection.ThisisforsafetyreasonstoensureconsistencyofIDs.

Restoringcollectionswithshardingprototypes

arangorestorewillyieldanerror,whiletryingtorestoreacollection,whosesharddistributionfollowsacollection,whichdoesnotexistintheclusterandwhichwasnotdumpedalong:

unix>arangorestore--collectionclonedCollection--server.databasemydb--input-directory"dump"

ERRORgoterrorfromserver:HTTP500(InternalServerError):ArangoError1486:mustnothaveadistributeShardsLikeattribute

pointingtoanunknowncollection

Processed0collection(s),read0byte(s)fromdatafiles,sent0batch(es)

Thecollectioncanberestoredbyoverridingtheerrormessageasfollows:

unix>arangorestore--collectionclonedCollection--server.databasemydb--input-directory"dump"--ignore-distribute-shards-li

ke-errors

RestoreintoanauthenticationenabledArangoDB

OfcourseyoucanrestoredataintoapasswordprotectedArangoDBaswell.Howeverthisrequirescertainuserrightsfortheuserusedintherestoreprocess.TherightsaredescribedindetailintheManagingUserschapter.Forrestorethisshortoverviewissufficient:

Whenimportingintoanexistingdatabase,thegivenuserneedsAdministrateaccessonthisdatabase.WhencreatinganewDatabaseduringrestore,thegivenuserneedsAdministrateaccesson_system.TheuserwillbepromotedwithAdministrateaccessonthenewlycreateddatabase.

Arangorestore

ExportingDatafromanArangoDBdatabaseToexportdatafromanArangoDBserverinstance,youwillneedtoinvokearangoexport.arangoexportcanbeinvokedbyexecutingthefollowingcommand:

unix>arangoexport--collectiontest--output-directory"dump"

Thisexportsthecollectionstestintothedirectorydumpasonebigjsonarray.Everyentryinthisarrayisonedocumentfromthecollectionwithoutaspecificorder.Toexportmorethanonecollectionatatimespecifymultiple--collectionoptions.

Thedefaultoutputdirectoryisexport.

arangoexportwillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabaseoradifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:

Here'sanexampleofexportingdatafromanon-standardendpoint,usingadedicateddatabasename:

unix>arangoexport--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--collectiontes

t--output-directory"my-export"

Whenfinished,arangoexportwillprintoutasummarylinewithsomeaggregatestatisticsaboutwhatitdid,e.g.:

Processed2collection(s),wrote9031763Byte(s),78HTTPrequest(s)

ExportJSON

unix>arangoexport--typejson--collectiontest

Thisexportsthecollectiontestintotheoutputdirectoryexportasonejsonarray.Everyarrayentryisonedocumentfromthecollectiontest

ExportJSONL

unix>arangoexport--typejsonl--collectiontest

Thisexportsthecollectiontestintotheoutputdirectoryexportasjsonl.Everylineintheexportisonedocumentfromthecollectiontestasjson.

ExportCSV

unix>arangoexport--typecsv--collectiontest--fields_key,_id,_rev

ThisexportsthecollectiontestintotheoutputdirectoryexportasCSV.Thefirstlinecontainstheheaderwithallfieldnames.EachlineisonedocumentrepresentedasCSVandseparatedwithacomma.ObjectsandArraysarerepresentedasaJSONstring.

Arangoexport

ExportXML

unix>arangoexport--typexml--collectiontest

ThisexportsthecollectiontestintotheoutputdirectoryexportasgenericXML.TherootelementofthegeneratedXMLfileisnamedcollection.EachdocumentinthecollectionisexportedinadocXMLattribute.Eachdocumentattributeisexportinagenericattelement,whichhasatypeattributeindicatingtheattributevalue,andavalueattributecontainingtheattribute'svalue.

ExportXGMMLXGMMLisanXMLapplicationbasedonGML.ToviewtheXGMMLfileyoucanuseforexampleCytoscape.

importantnoteIfyouexportallattributes(--xgmml-label-onlyfalse)keepinmindthataatrributenamestypehavetobethesametypeforalldocuments.Itwontworkifyouhaveaattributenamedrankthatisinonedocumentastringandinanotherdocumentainteger.

//doc1

"rank":1

//doc2

"rank":"2"

//doc1

"rank":1

//doc2

"rank":2

XGMMLspecificoptions--xgmml-label-attributespecifythenameoftheattributethatwillbecomethelabelinthexgmmlfile.

--xgmml-label-onlysettotruewillonlyexportthelabelwithoutanyattributesinedgesornodes.

exportbasedoncollections

unix>arangoexport--typexgmml--graph-namemygraph--collectionvertex--collectionedge

Thisexportstheaunnamedgraphwithvertexcollectionvertexandedgecollectionedgeintothexgmmlfilemygraph.xgmml.

exportbasedonanamedgraph

unix>arangoexport--typexgmml--graph-namemygraph

Arangoexport

Thisexportsthenamedgraphmygraphintothexgmmlfilemygraph.xgmml.

exportXGMMLwithoutattributes

unix>arangoexport--typexgmml--graph-namemygraph--xgmml-label-onlytrue

Thisexportsthenamedgraphmygraphintothexgmmlfilemygraph.xgmmlwithoutthe<att>taginnodesandedges.

exportXGMMLwithaspecificlabel

unix>arangoexport--typexgmml--graph-namemygraph--xgmml-label-attributename

Thisexportsthenamedgraphmygraphintothexgmmlfilemygraph.xgmmlwithalabelfromdocumentsattributenameinsteadofthedefaultattributelabel.

ExportviaAQLquery

unix>arangoexport--typejsonl--query"forbookinbooksfilterbook.sells>100returnbook"

Exportviaanaqlqueryallowsyoutoexportthereturneddataasthetypespecifiedwith--type.Theexampleexportsallbooksasjsonlthataresoldmorethan100times.

Arangoexport

ManagingUsersTheusermanagementinArangoDB3issimilartotheonesfoundinMySQL,PostgreSQL,orotherdatabasesystems.

Usermanagementispossibleinthewebinterfaceandinarangoshwhileloggedontothe_systemdatabase.

Notethattheonlyusernamesmustnotstartwith:role:.

ActionsandAccessLevelsAnArangoDBservercontainsalistofusers.Italsodefinesvariousaccesslevelsthatcanbeassignedtoauser(fordetails,seebelow)andthatareneededtoperformcertainactions.Theseactionscanbegroupedintothreecategories:

serveractionsdatabaseactionscollectionactions

Theserveractionsare

createuser:allowstocreateanewuser.

updateuser:allowstochangetheaccesslevelsanddetailsofanexistinguser.

dropuser:allowstodeleteanexistinguser.

createdatabase:allowstocreateanewdatabase.

dropdatabase:allowstodeleteanexistingdatabase.

shutdownserver:removeserverfromclusterandshutdown

Thedatabaseactionsaretiedtoagivendatabase,andaccesslevelsmustbesetforeachdatabaseindividually.Foragivendatabasetheactionsare

createcollection:allowstocreateanewcollectioninthegivendatabase.

updatecollection:allowstoupdatepropertiesofanexistingcollection.

dropcollection:allowstodeleteanexistingcollection.

createindex:allowstocreateanindexforanexistingcollectioninthegivendatabase.

dropindex:allowstodeleteanindexofanexistingcollectioninthegivendatabase.

Thecollectionactionsaretiedtoagivencollectionofagivendatabase,andaccesslevelsmustbesetforeachcollectionindividually.Foragivencollectiontheactionsare

readdocument:readadocumentofthegivencollection.

createdocument:createsanewdocumentinthegivencollection.

modifydocument:modifiesanexistingdocumentofthegivencollection,thiscanbeanupdateorreplaceoperation.

dropdocument:deletesanexistingdocumentofthegivencollection.

truncatecollection:deletesalldocumentsofagivencollection.

Toperformactionsontheserverleveltheuserneedsatleastthefollowingaccesslevels.TheaccesslevelsareAdministrateandNoaccess:

serveraction serverlevel

createadatabase Administrate

dropadatabase Administrate

createauser Administrate

ManagingUsers

updateauser Administrate

updateuseraccesslevel Administrate

dropauser Administrate

shutdownserver Administrate

Toperformactionsinaspecificdatabase(likecreatingordroppingcollections),auserneedsatleastthefollowingaccesslevel.ThepossibleaccesslevelsfordatabasesareAdministrate,AccessandNoaccess.TheaccesslevelsforcollectionsareRead/Write,ReadOnlyandNoAccess.

databaseaction databaselevel collectionlevel

createcollection Administrate Read/Write

listcollections Access ReadOnly

renamecollection Administrate Read/Write

modifycollectionproperties Administrate Read/Write

readproperties Access ReadOnly

dropcollection Administrate Read/Write

createanindex Administrate Read/Write

dropanindex Administrate Read/Write

seeindexdefinition Access ReadOnly

NotethattheaccesslevelAccessforadatabaseisalwaysrequiredtoperformanyactiononacollectioninthatdatabase.

Forcollectionsauserneedsthefollowingaccesslevelstothegivendatabaseandthegivencollection.TheaccesslevelsforthedatabaseareAdministrate,AccessandNoaccess.TheaccesslevelsforthecollectionareRead/Write,ReadOnlyandNoAccess.

action collectionlevel databaselevel

readadocument Read/WriteorReadOnly AdministrateorAccess

createadocument Read/Write AdministrateorAccess

modifyadocument Read/Write AdministrateorAccess

dropadocument Read/Write AdministrateorAccess

truncateacollection Read/Write AdministrateorAccess

Example

Forexample,given

adatabaseexampleacollectiondatainthedatabaseexampleauserJohnSmith

IftheuserJohnSmithisassignedtheaccesslevelAccessforthedatabaseexampleandthelevelRead/Writeforthecollectiondata,thentheuserisallowedtoread,create,modifyordeletedocumentsinthecollectiondata.Buttheuseris,forexample,notallowedtocreateindexesforthecollectiondatanorcreatenewcollectionsinthedatabaseexample.

GrantingAccessLevels

Accesslevelscanbemanagedviathewebinterfaceorinarangosh.

Inordertograntanaccessleveltoauser,youcanassignoneofthreeaccesslevelsforeachdatabaseandoneofthreelevelsforeachcollectioninadatabase.Theserveraccesslevelfortheuserfollowsfromthedatabaseaccesslevelinthe_systemdatabase,itisAdministrateifandonlyifthedatabaseaccesslevelisAdministrate.NotethatthismeansthatdatabaseaccesslevelAccessdoesnotgrantauserserveraccesslevelAdministrate.

ManagingUsers

InitialAccessLevels

WhenausercreatesadatabasetheaccessleveloftheuserforthatdatabaseissettoAdministrate.Thesameistrueforcreatingacollection,inthiscasetheusergetRead/Writeaccesstothecollection.

WildcardDatabaseAccessLevel

Withtheabovedefinition,onemustdefinethedatabaseaccesslevelforalldatabase/userpairsintheserver,whichwouldbeverytedious.Inordertosimplifythisprocess,itispossibletodefine,forauser,awildcarddatabaseaccesslevel.Thiswildcardisusedifthedatabaseaccesslevelisnotexplicitlydefinedforacertaindatabase.EachnewcreateduserhasaninitialdatabasewildcardofNoAccess.

Changingthewildcarddatabaseaccesslevelforauserwillchangetheaccesslevelforalldatabasesthathavenoexplicitlydefinedaccesslevel.Notethatthisincludesdatabaseswhichwillbecreatedinthefutureandforwhichnoexplicitaccesslevelsaresetforthatuser!

Ifyoudeletethewildcard,thedefaultaccesslevelisdefinedasNoAccess.

TherootuserhasaninitialdatabasewildcardofAdministrate.

Example

AssumeuserJohnSmithhasthefollowingdatabaseaccesslevels:

accesslevel

database* Access

databaseshop1 Administrate

databaseshop2 NoAccess

ThiswillgivetheuserJohnSmiththefollowingdatabaselevelaccess:

databaseshop1:Administratedatabaseshop2:NoAccessdatabasesomething:Access

Ifthewildcard*ischangedfromAccesstoNoAccessthenthepermissionswillchangeasfollows:

databaseshop1:Administratedatabaseshop2:NoAccessdatabasesomething:NoAccess

WildcardCollectionAccessLevel

Foreachuseranddatabasethereisawildcardcollectionaccesslevel.Thislevelisusedforallcollectionspairswithoutanexplicitlydefinedcollectionaccesslevel.Notethatthisincludescollectionswhichwillbecreatedinthefutureandforwhichnoexplicitaccesslevelsaresetforathatuser!EachnewcreateduserhasaninitialcollectionwildcardofNoAccess.

Ifyoudeletethewildcard,thesystemdefaultstoNoAccess.

TherootuserhasaninitialcollectionwildcardofRead/Writeineverydatabase.

Whencreatingauserthroughdb._createDatabase(name,options,users)theaccessleveloftheuserforthisdatabasewillbesettoAdministrateandthewildcardforallcollectionswithinthisdatabasewillbesettoRead/Write.

Example

AssumeuserJohnSmithhasthefollowingdatabaseaccesslevels:

accesslevel

database* Access

andcollectionaccesslevels:

accesslevel

ManagingUsers

database*,collection* Read/Write

databaseshop1,collectionproducts Read-Only

databaseshop1,collection* NoAccess

databaseshop2,collection* Read-Only

Thentheuserdoewillgetthefollowingcollectionaccesslevels:

databaseshop1,collectionproducts:Read-Onlydatabaseshop1,collectioncustomers:NoAccessdatabaseshop2,collectionreviews:Read-Onlydatabasesomething,collectionelse:Read/Write

Explanation:

Databaseshop1,collectionproductsdirectlymatchesadefinedaccesslevel.ThislevelisdefinedasRead-Only.

Databaseshop1,collectioncustomersdoesnotmatchadefinedaccesslevel.However,databaseshop1matchesandthewildcardinthisdatabaseforcollectionlevelisNoAccess.

Databaseshop2,collectionreviewsdoesnotmatchadefinedaccesslevel.However,databaseshop2matchesandthewildcardinthisdatabaseforcollectionlevelisRead-Only.

Databasesomehing,collectionelsedoesnotmatchadefinedaccesslevel.Thedatabasesomethingalsodoeshaveadirectmatches.Thereforethewildcardisselected.ThelevelisRead/Write.

SystemCollections

Theaccesslevelforsystemcollectionscannotbechanged.Theyfollowdifferentrulesthanuserdefinedcollectionsandmaychangewithoutfurthernotice.Currentlythesystemcollectionsfollowtheserules:

collection accesslevel

_users(in_system) NoAccess

_queues Read-Only

_frontend Read/Write

* sameasdb

AllothersystemcollectionshaveaccesslevelRead/WriteiftheuserhasAdministrateaccesstothedatabase.TheyhaveaccesslevelRead/OnlyiftheuserhasAccesstothedatabase.

TomodifythesesystemcollectionsyoushouldalwaysusethespecializedAPIsprovidedbyArangoDB.Forexamplenouserhasaccesstothe_userscollectioninthe_systemdatabase.Allchangestotheaccesslevelsmustbedoneusingthe@arangodb/usersmodule,the/_users/APIorthewebinterface.

ManagingUsers

ManagingUsersintheArangoDBShellPleasenote,thatforbackwardcompatibilitytheserveraccesslevelsfollowfromthedatabaseaccesslevelonthedatabase_system.

Alsonotethattheserveranddatabaseaccesslevelsarerepresentedas

rw:forAdministratero:forAccessnone:forNoaccess

Thisisagainforbackwardcompatibility.

Example

Fireuparangoshandrequiretheusersmodule.Useittocreateanewuser:

arangosh>varusers=require('@arangodb/users');

arangosh>users.save('JohnSmith','mypassword');

CreatesausercalledJohnSmith.Thisuserwillhavenoaccessatall.

arangosh>users.grantDatabase('JohnSmith','testdb','rw');

ThisgrantstheuserAdministrateaccesstothedatabasetestdb.revokeDatabasewillrevokethisaccesslevelsetting.

Note:Beawarethatfrom3.2onwardsthegrantDatabasewillnotautomaticallygrantuserstheaccessleveltowriteorreadcollectionsinadatabase.IfyougrantaccesstoadatabasetestdbyouwilladditionallyneedtoexplicitlygrantaccesslevelstoindividualcollectionsviagrantCollection.

Theupgradeprocedurefrom3.1to3.2setsthewildcarddatabaseaccesslevelforalluserstoAdministrateandsetsthewildcardcollectionaccesslevelforalluser/databasepairstoRead/Write.

arangosh>users.grantCollection('JohnSmith','testdb','testcoll','rw');

Saveusers.save(user,passwd,active,extra)

ThiswillcreateanewArangoDBuser.Theusernamemustbespecifiedinuserandmustnotbeempty.

Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.IfyoupassthespecialvalueARANGODB_DEFAULT_ROOT_PASSWORD,thepasswordwillbesetthevaluestoredintheenvironmentvariableARANGODB_DEFAULT_ROOT_PASSWORD.ThiscanbeusedtopassaninstancevariableintoArangoDB.Forexample,theinstanceidentifierfromAmazon.

Iftheactiveattributeisnotspecified,itdefaultstotrue.Theextraattributecanbeusedtosavecustomdatawiththeuser.

Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,ortherealreadyexistsauserwiththespecifiedname.

Note:Theuserwillnothavepermissiontoaccessanydatabase.YouneedtogranttheaccessrightsforoneormoredatabasesusinggrantDatabase.

Examples

arangosh>require('@arangodb/users').save('my-user','my-secret-password');

InArangosh

GrantDatabaseusers.grantDatabase(user,database,type)

Thisgrantstype('rw','ro'or'none')accesstothedatabasefortheuser.Ifdatabaseis"*",thissetsthewildcarddatabaseaccesslevelfortheuseruser.

Theserveraccesslevelfollowsfromtheaccesslevelforthedatabase_system.

RevokeDatabaseusers.revokeDatabase(user,database)

Thisclearstheaccesslevelsettingtothedatabasefortheuserandthewildcarddatabaseaccesssettingforthisuserkicksin.IncasenowildcardaccesswasdefinedthedefaultisNoAccess.Thiswillalsocleartheaccesslevelsforallthecollectionsinthisdatabase.

GrantCollectionusers.grantCollection(user,database,collection,type)

Thisgrantstype('rw','ro'or'none')accessleveltothecollectionindatabasefortheuser.Ifcollectionis"*"thissetsthewildcardcollectionaccesslevelfortheuseruserindatabasedatabase.

RevokeCollectionusers.revokeCollection(user,database)

Thisclearstheaccesslevelsettingtothecollectioncollectionfortheuseruser.ThesystemwilleitherfallbacktothewildcardcollectionaccesslevelordefaulttoNoAccess

Replaceusers.replace(user,passwd,active,extra)

ThiswilllookupanexistingArangoDBuserandreplaceitsuserdata.

Theusernamemustbespecifiedinuser,andauserwiththespecifiednamemustalreadyexistinthedatabase.

Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.

Iftheactiveattributeisnotspecified,itdefaultstotrue.Theextraattributecanbeusedtosavecustomdatawiththeuser.

Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,orifthespecifiedusercannotbefoundinthedatabase.

Note:thisfunctionwillnotworkfromwithinthewebinterface

Examples

arangosh>require("@arangodb/users").replace("my-user","my-changed-password");

Updateusers.update(user,passwd,active,extra)

ThiswillupdateanexistingArangoDBuserwithanewpasswordandotherdata.

Theusernamemustbespecifiedinuserandtheusermustalreadyexistinthedatabase.

InArangosh

Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.

Iftheactiveattributeisnotspecified,thecurrentvaluesavedfortheuserwillnotbechanged.Thesameistruefortheextraattribute.

Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,orifthespecifiedusercannotbefoundinthedatabase.

Examples

arangosh>require("@arangodb/users").update("my-user","my-secret-password");

isValidusers.isValid(user,password)

Checkswhetherthegivencombinationofusernameandpasswordisvalid.Thefunctionwillreturnabooleanvalueifthecombinationofusernameandpasswordisvalid.

Eachcalltothisfunctionispenalizedbytheserversleepingarandomamountoftime.

Examples

arangosh>require("@arangodb/users").isValid("my-user","my-secret-password");

Removeusers.remove(user)

RemovesanexistingArangoDBuserfromthedatabase.

TheusernamemustbespecifiedinUserandthespecifiedusermustexistinthedatabase.

Thismethodwillfailiftheusercannotbefoundinthedatabase.

Examples

arangosh>require("@arangodb/users").remove("my-user");

Documentusers.document(user)

FetchesanexistingArangoDBuserfromthedatabase.

Theusernamemustbespecifiedinuser.

Examples

arangosh>require("@arangodb/users").document("my-user");

InArangosh

users.all()

FetchesallexistingArangoDBusersfromthedatabase.

Examples

arangosh>require("@arangodb/users").all();

Reloadusers.reload()

Reloadstheuserauthenticationdataontheserver

Alluserauthenticationdataisloadedbytheserveronceonstartuponlyandiscachedafterthat.Whenusersgetaddedordeleted,acacheflushisdoneautomatically,andthiscanbeperformedbyacalltothismethod.

Examples

arangosh>require("@arangodb/users").reload();

Permissionusers.permission(user,database[,collection])

Fetchestheaccessleveltothedatabaseoracollection.

Theuseranddatabasenamemustbespecified,optionallyyoucanspecifythecollectionname.

Examples

arangosh>require("@arangodb/users").permission("my-user","testdb");

InArangosh

Command-lineoptions

GeneralOptions

Generalhelp--help

Printsalistofthemostcommonoptionsavailableandthenexits.Inordertoseealloptionsuse--help-all.

Version--version

Printstheversionoftheserverandexits.

ConfigurationFiles

Optionscanbespecifiedonthecommandlineorinconfigurationfiles.IfastringVariableoccursinthevalue,itisreplacedbythecorrespondingenvironmentvariable.

--configurationfilename

-cfilename

Specifiesthenameoftheconfigurationfiletouse.

Ifthiscommandisnotpassedtotheserver,thenbydefault,theserverwillattempttofirstlocateafilenamed~/.arango/arangod.confintheuser'shomedirectory.

Ifnosuchfileisfound,theserverwillproceedtolookforafilearangod.confinthesystemconfigurationdirectory.Thesystemconfigurationdirectoryisplatform-specific,andmaybechangedwhencompilingArangoDByourself.Itmaydefaultto/etc/arangodbor/usr/local/etc/arangodb.Thisfileisinstalledwhenusingapackagemanagerlikerpmordpkg.IfyoumodifythisfileandlaterupgradetoanewversionofArangoDB,thenthepackagemanagernormallywarnsyouabouttheconflict.Inordertoavoidthesewarningforsmalladjustments,youcanputlocaloverridesintoafilearangod.conf.local.

Onlycommandlineoptionswithavalueshouldbesetwithintheconfigurationfile.Commandlineoptionswhichactasflagsshouldbeenteredonthecommandlinewhenstartingtheserver.

Eachoptionisspecifiedonaseparatelineintheform:

key=value

Alternatively,aheadersectioncanbespecifiedandoptionspertainingtothatsectioncanbespecifiedinashorterform

level=trace

ratherthanspecifying

log.level=trace

Soyouseeingeneral--section.paramvaluetranslatesto

[section]

param=value

ServerConfiguration

Whitespacearound=isignoredintheconfigurationfile.Donotputspacesaroundadditional=intheparametervaluehowever.Thefollowingexampleshowsthecorrectwaytospecifyalogleveloftraceforthetopicstartup:

log.level=startup=trace

Notethatthereisnowhitespacebetweenstartupand=,andalsonot=andtrace.

Whereonesectionmayoccurmultipletimes,andthelastoccuranceofparamwillbecomethefinalvalue.Incaseofparametersbeingvectors,multipleoccuranceaddsanotheritemtothevector.Vectorscanbeidentifiedbythe...inthe--helpoutputofthebinaries.

Commentscanbeplacedintheconfigurationfile,onlyifthelinebeginswithoneormorehashsymbols(#).

Theremaybeoccasionswhereaconfigurationfileexistsandtheuserwishestooverrideconfigurationsettingsstoredinaconfigurationfile.Anysettingsspecifiedonthecommandlinewilloverwritethesamesettingwhenitappearsinaconfigurationfile.Iftheuserwishestocompletelyignoreconfigurationfileswithoutnecessarilydeletingthefile(orfiles),thenaddthecommandlineoption

-cnone

--configurationnone

whenstartinguptheserver.Notethat,thewordnoneiscase-insensitive.

ServerConfiguration

FileSystems

(LINUX)

WerecommendtonotuseBTRFSonlinux,it'sknowntonotworkwellinconjunctionwithArangoDB.WeexperiencedthatarangodbfacinglatencyissuesonaccessingitsdatabasefilesonBTRFSpartitions.InconjunctionwithBTRFSandAUFSwealsosawdatalossonrestart.

VirtualMemoryPageSizes(LINUX)

Bydefault,ArangoDBusesJemallocasthememoryallocator.Jemallocdoesagoodjobofreducingvirtualmemoryfragmentation,especiallyforlong-runningprocesses.Unfortunately,someOSconfigurationscaninterferewithJemalloc'sabilitytofunctionproperly.Specifically,Linux's"transparenthugepages",Windows'"largepages"andothersimilarfeaturessometimespreventJemallocfromreturningunusedmemorytotheoperatingsystemandresultinunnecessarilyhighmemoryuse.Therefore,werecommenddisablingthesefeatureswhenusingJemallocwithArangoDB.Pleaseconsultyouroperatingsystem'sdocumentationforhowtodothis.

Execute

sudobash-c"echomadvise>/sys/kernel/mm/transparent_hugepage/enabled"

sudobash-c"echomadvise>/sys/kernel/mm/transparent_hugepage/defrag"

beforeexecutingarangod.

SwapSpace

(LINUX)

Itisrecommendedtoassignswapspaceforaserverthatisrunningarangod.Configuringswapspacecanpreventtheoperatingsystem'sOOMkillerfromkillingArangoDBtooeagerlyonLinux.

Over-CommitMemory

FortheMMFilesstorageengine,execute

sudobash-c"echo0>/proc/sys/vm/overcommit_memory"

FortheRocksDBstorageengine,execute

sudobash-c"echo2>/proc/sys/vm/overcommit_memory"

beforestarting.

Fromwww.kernel.org:

Whenthisflagis0,thekernelattemptstoestimatetheamountoffreememoryleftwhenuserspacerequestsmorememory.

Whenthisflagis1,thekernelpretendsthereisalwaysenoughmemoryuntilitactuallyrunsout.

Whenthisflagis2,thekernelusesa"neverovercommit"policythatattemptstopreventanyovercommitofmemory.

Notethatthenusinganovercommit_memorysettingof2,thiswillbydefaultallowprocessestouseallswapspacebutonlyhalfoftheavailableRAM.Thiscanbechangedbyadjustingthevalueofovercommit_ratioaswell.

Fromwww.kernel.org:

Whenovercommit_memoryissetto2,thecommittedaddressspaceisnotpermittedtoexceedswapplusthispercentageofphysicalRAM.

ZoneReclaim

Execute

sudobash-c"echo0>/proc/sys/vm/zone_reclaim_mode"

Fromwww.kernel.org:

ThisisvalueORedtogetherof

1=Zonereclaimon2=Zonereclaimwritesdirtypagesout4=Zonereclaimswapspages

NUMAMulti-processorsystemsoftenhavenon-uniformAccessMemory(NUMA).ArangoDBshouldbestartedwithinterleaveonsuchsystem.Thiscanbeachievedusing

numactl--interleave=allarangod...

MaxMemoryMappings(LINUX)

Linuxkernelsbydefaultrestrictthemaximumnumberofmemorymappingsofasingleprocesstoabout64Kmappings.Whilethisvalueissufficientformostworkloads,itmaybetoolowforaprocessthathaslotsofparallelthreadsthatallrequiretheirownmemorymappings.Inthiscaseallthethreads'memorymappingswillbeaccountedtothesinglearangodprocess,andthemaximumnumberof64Kmappingsmaybereached.Whenthemaximumnumberofmappingsisreached,callstommapwillfail,sotheprocesswillthinknomorememoryisavailablealthoughtheremaybeplentyofRAMleft.

Toavoidthisscenario,itisrecommendedtoraisethedefaultvalueforthemaximumnumberofmemorymappingstoasufficientlyhighvalue.Asaruleofthumb,onecoulduse8timesthenumberofavailablecorestimes8,000.

Fora32coreserver,agoodrule-of-thumbvaluethuswouldbe2,048,000(3288000).Tosetthevalueonce,usethefollowingcommandbeforestartingarangod:

sudobash-c"sysctl-w'vm.max_map_count=2048000'"

Tomakethesettingsdurable,itwillbenecessarytostoretheadjustedsettingsin/etc/sysctl.conforotherplacesthattheoperatingsystemislookingat.

EnvironmentVariables(LINUX)

ItisrecommendedtosettheenvironmentvariableGLIBCXX_FORCE_NEWto1onsystemsthatuseglibc++inordertodisablethememorypoolingbuiltintoglibc++.ThatmemorypoolingisunnecessarybecauseJemallocwillalreadydomemorypooling.

Execute

exportGLIBCXX_FORCE_NEW=1

beforestartingarangod.

32bitWhileitispossibletocompileArangoDBon32bitsystem,thisisnotarecommendedenvironment.64bitsystemscanaddressasignificantlybiggermemoryregion.

ManagingEndpointsTheArangoDBservercanlistenforincomingrequestsonmultipleendpoints.

TheendpointsarenormallyspecifiedeitherinArangoDB'sconfigurationfileoronthecommand-line,usingthe--server.endpoint.ArangoDBsupportsdifferenttypesofendpoints:

tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryptionssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryptionunix:///path/to/socket-Unixdomainsocketendpoint

IfaTCP/IPendpointisspecifiedwithoutaportnumber,thenthedefaultport(8529)willbeused.Ifmultipleendpointsneedtobeused,theoptioncanberepeatedmultipletimes.

ThedefaultendpointforArangoDBistcp://127.0.0.1:8529ortcp://localhost:8529.

EXAMPLES

unix>./arangod--server.endpointtcp://127.0.0.1:8529

--server.endpointssl://127.0.0.1:8530

--ssl.keyfileserver.pem/tmp/vocbase

2012-07-26T07:07:47Z[8161]INFOusingSSLprotocolversion'TLSv1'

2012-07-26T07:07:48Z[8161]INFOusingendpoint'ssl://127.0.0.1:8530'forhttpsslrequests

2012-07-26T07:07:48Z[8161]INFOusingendpoint'tcp://127.0.0.1:8529'forhttptcprequests

2012-07-26T07:07:49Z[8161]INFOArangoDB(version1.1.alpha)isreadyforbusiness

2012-07-26T07:07:49Z[8161]INFOHaveFun!

TCPEndpoints

Givenahostname:

--server.endpointtcp://hostname:port

GivenanIPv4address:

--server.endpointtcp://ipv4-address:port

GivenanIPv6address:

--server.endpointtcp://[ipv6-address]:port

Ononespecificethernetinterfaceeachportcanonlybeboundexactlyonce.YoucanlookupyouravailableinterfacesusingtheifconfigcommandonLinux/MacOSX-theWindowsequivalentisipconfig(SeeWikipediaformoredetails).ThegeneralnamesoftheinterfacesdifferonOS'sandhardwarestheyrunon.However,typicallyeveryhosthasasocalledloopbackinterface,whichisavirtualinterface.Byconventionitalwayshastheaddress127.0.0.1or::1(ipv6),andcanonlybereachedfromexactlytheverysamehost.Ethernetinterfacesusuallyhavenameslikeeth0,wlan0,eth1:17,le0oraplaintextnameinWindows.

Tofindoutwhichservicesalreadyuseports(soArangoDBcan'tbindthemanymore),youcanusethenetstatcommand(itbehavesalittledifferentoneachplatform,runitwith-lnptonLinux,-ptcponMacOSXorwith-anonwindowsforvaluableinformation).

ArangoDBcanalsodoasocalledbroadcastbindusingtcp://0.0.0.0:8529.Thiswayitwillbereachableonallinterfacesofthehost.Thismaybeusefulondevelopmentsystemsthatfrequentlychangetheirnetworksetuplikelaptops.

SpecialnoteonIPv6link-localaddresses

ArangoDBcanalsolistentoIPv6link-localaddressesviaaddingthezoneIDtotheIPv6addressintheform[ipv6-link-local-address%zone-id].However,whatyouprobablyinsteadwantistobindtoalocalIPv6address.LocalIPv6addressesstartwithfd.Ifyouonlyseeafe80:IPv6addressinyourinterfaceconfigurationbutnoIPv6addressstartingwithfdyourinterfacehasnolocalIPv6addressassigned.YoucanreadmoreaboutIPv6link-localaddresseshere.

Example

ManagingEndpoints

Bindtoalink-localandlocalIPv6address.

unix>ifconfig

Thiscommandlistsallinterfacesandassignedipaddresses.Thelink-localaddressmaybefe80::6257:18ff:fe82:3ec6%eth0(IPv6addressplusinterfacename).AlocalIPv6addressmaybefd12:3456::789a.TobindArangoDBtoitstartarangodwith--server.endpointtcp://[fe80::6257:18ff:fe82:3ec6%eth0]:8529.Usetelnettotesttheconnection.

unix>telnetfe80::6257:18ff:fe82:3ec6%eth08529

Tryingfe80::6257:18ff:fe82:3ec6...

Connectedtomy-machine.

Escapecharacteris'^]'.

GET/HTTP/1.1

HTTP/1.1301MovedPermanently

Location:/_db/_system/_admin/aardvark/index.html

Content-Type:text/html

Server:ArangoDB

Connection:Keep-Alive

Content-Length:197

<html><head><title>Moved</title></head><body><h1>Moved</h1><p>Thispagehasmovedto<ahref="/_db/_system/_admin/aardvark/inde

x.html">/_db/_system/_admin/aardvark/index.html</a>.</p></body></html>

Reuseaddress--tcp.reuse-address

IfthisbooleanoptionissettotruethenthesocketoptionSO_REUSEADDRissetonallserverendpoints,whichisthedefault.Ifthisoptionissettofalseitispossiblethatittakesuptoaminuteafteraserverhasterminateduntilitispossibleforanewservertousethesameendpointagain.Thisiswhythisisactivatedbydefault.

Pleasenotehoweverthatundersomeoperatingsystemsthiscanbeasecurityriskbecauseitmightbepossibleforanotherprocesstobindtothesameaddressandport,possiblyhijackingnetworktraffic.UnderWindows,ArangoDBadditionallysetstheflagSO_EXCLUSIVEADDRUSEasameasuretoalleviatethisproblem.

Backlogsize--tcp.backlog-size

AllowstospecifythesizeofthebacklogforthelistensystemcallThedefaultvalueis10.Themaximumvalueisplatform-dependent.Specifyingahighervaluethandefinedinthesystemheader'sSOMAXCONNmayresultinawarningonserverstart.Theactualvalueusedbylistenmayalsobesilentlytruncatedonsomeplatforms(thishappensinsidethelistensystemcall).

ManagingEndpoints

SSLConfiguration

SSLEndpoints

Givenahostname:

--server.endpointtcp://hostname:port

GivenanIPv4address:

--server.endpointtcp://ipv4-address:port

GivenanIPv6address:

--server.endpointtcp://[ipv6-address]:port

Note:IfyouareusingSSL-encryptedendpoints,youmustalsosupplythepathtoaservercertificateusingthe--ssl.keyfileoption.

Keyfile--ssl.keyfilefilename

IfSSLencryptionisused,thisoptionmustbeusedtospecifythefilenameoftheserverprivatekey.ThefilemustbePEMformattedandcontainboththecertificateandtheserver'sprivatekey.

Thefilespecifiedbyfilenamecanbegeneratedusingopenssl:

#createprivatekeyinfile"server.key"

opensslgenrsa-des3-outserver.key1024

#createcertificatesigningrequest(csr)infile"server.csr"

opensslreq-new-keyserver.key-outserver.csr

#copyawayoriginalprivatekeyto"server.key.org"

cpserver.keyserver.key.org

#removepassphrasefromtheprivatekey

opensslrsa-inserver.key.org-outserver.key

#signthecsrwiththekey,createscertificatePEMfile"server.crt"

opensslx509-req-days365-inserver.csr-signkeyserver.key-outserver.crt

#combinecertificateandkeyintosinglePEMfile"server.pem"

catserver.crtserver.key>server.pem

YoumayusecertificatesissuedbyaCertificateAuthorityorself-signedcertificates.Self-signedcertificatescanbecreatedbyatoolofyourchoice.WhenusingOpenSSLforcreatingtheself-signedcertificate,thefollowingcommandsshouldcreateavalidkeyfile:

-----BEGINCERTIFICATE-----

(base64encodedcertificate)

-----ENDCERTIFICATE-----

-----BEGINRSAPRIVATEKEY-----

(base64encodedprivatekey)

-----ENDRSAPRIVATEKEY-----

Forfurtherinformationpleasecheckthemanualsofthetoolsyouusetocreatethecertificate.

CAFile--ssl.cafilefilename

SSLConfiguration

ThisoptioncanbeusedtospecifyafilewithCAcertificatesthataresenttotheclientwhenevertheserverrequestsaclientcertificate.Ifthefileisspecified,TheserverwillonlyacceptclientrequestswithcertificatesissuedbytheseCAs.Donotspecifythisoptionifyouwantclientstobeabletoconnectwithoutspecificcertificates.

ThecertificatesinfilenamemustbePEMformatted.

SSLprotocol

--ssl.protocolvalue

Usethisoptiontospecifythedefaultencryptionprotocoltobeused.Thefollowingvariantsareavailable:

1:SSLv22:SSLv2orSSLv3(negotiated)3:SSLv34:TLSv15:TLSv1.2

Thedefaultvalueis5(TLSv1.2).

SSLcache--ssl.session-cachevalue

SettotrueifSSLsessioncachingshouldbeused.

valuehasadefaultvalueoffalse(i.e.nocaching).

SSLpeercertificate

ThisfeatureisavailableintheEnterpriseEdition.

--ssl.require-peer-certificate

Requireapeercertificatefromtheclientbeforeconnecting.

SSLoptions

--ssl.optionsvalue

ThisoptioncanbeusedtosetvariousSSL-relatedoptions.IndividualoptionvaluesmustbecombinedusingbitwiseOR.

WhichoptionsareavailableonyourplatformisdeterminedbytheOpenSSLversionyouuse.Thelistofoptionsavailableonyourplatformmightberetrievedbythefollowingshellcommand:

>grep"#defineSSL_OP_.*"/usr/include/openssl/ssl.h

#defineSSL_OP_MICROSOFT_SESS_ID_BUG0x00000001L

#defineSSL_OP_NETSCAPE_CHALLENGE_BUG0x00000002L

#defineSSL_OP_LEGACY_SERVER_CONNECT0x00000004L

#defineSSL_OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG0x00000008L

#defineSSL_OP_SSLREF2_REUSE_CERT_TYPE_BUG0x00000010L

#defineSSL_OP_MICROSOFT_BIG_SSLV3_BUFFER0x00000020L

AdescriptionoftheoptionscanbefoundonlineintheOpenSSLdocumentation

SSLcipher--ssl.cipher-listcipher-list

ThisoptioncanbeusedtorestricttheservertocertainSSLciphersonly,andtodefinetherelativeusagepreferenceofSSLciphers.

Theformatofcipher-listisdocumentedintheOpenSSLdocumentation.

Tocheckwhichciphersareavailableonyourplatform,youmayusethefollowingshellcommand:

SSLConfiguration

>opensslciphers-v

ECDHE-RSA-AES256-SHASSLv3Kx=ECDHAu=RSAEnc=AES(256)Mac=SHA1

ECDHE-ECDSA-AES256-SHASSLv3Kx=ECDHAu=ECDSAEnc=AES(256)Mac=SHA1

DHE-RSA-AES256-SHASSLv3Kx=DHAu=RSAEnc=AES(256)Mac=SHA1

DHE-DSS-AES256-SHASSLv3Kx=DHAu=DSSEnc=AES(256)Mac=SHA1

DHE-RSA-CAMELLIA256-SHASSLv3Kx=DHAu=RSAEnc=Camellia(256)

Mac=SHA1

Thedefaultvalueforcipher-listis"ALL".

SSLConfiguration

LDAPThisfeatureisavailableintheEnterpriseEdition.

BasicsConcepts

Therearetwomodesofoperation:simpleauthandbind+search.

ThebasicoptionsforspecifyinghowtoaccesstheLDAPserverare--ldap.enabled,--ldap.tls,--ldap.port,--ldap.server.--ldap.serverand--ldap.portcanbereplacedby--ldap.url.Thedefaultfor--ldap.portis389.

Simpleauth

ArangoDBconnectstotheldapserverandauthenticateswiththeusernameandpasswordprovidedbytheAPIauthenticationrequest.IftheLDAPserversuccessfullyverifiesthepasswordthentheuserisauthenticated.

If--ldap.prefixand/or--ldap.suffixisprovided,thenthesimplemodeisselected.

Inordertoauthorizetheuserforoneormoredatabasestherearetwomodesofoperation:databaseattributeorroles.

Databaseattribute

Inthismode,anLDAPattributeoftheuserisusedtospecifytheaccesslevelswithinLDAP.Thedatabase/collectionaccesslevelsinArangoDBarenotused.

--ldap.permissions-attribute-namehastheformatdatabase-name=(*|rw|none)[,database-name=(*|rw|none)].

Example:

--ldap.enabledtrue--ldap.serverldap.company.com\

--ldap.permissions-attribute-namearangodbPermissions\

--ldap.prefixuid=\

--ldap.suffix,dc=company,dc=com

--ldap.prefixand--ldap.suffixbuildthedistinguishedname(DN).ArangoDBtriestoauthenticatewithprefix+ArangoDBusername+suffixagainsttheLDAPserverandsearchesforthedatabasepermissions.

dn:uid=fermi,dc=example,dc=com

arangodbPermissions:foo=none,bar=rw

ThiswillgiveAdministrateaccesstobarandNoAcesstofoo.Notethatthismethodsonlyallowstospecifydatabaseaccesslevels,notcollectionaccesslevels.

Inthismode,anLDAPattributeoftheuserisusedtospecifyoneormorerolesforthatusers.Thedatabase/collectionaccesslevelsfortheserolesdefinedinArangoDBarethenused.

Example:

--ldap.roles-attribute-namegroupMembership\

--ldap.prefixuid=\

--ldap.prefixand--ldap.suffixbuildthedistinguishedname(DN).ArangoDBtrystoauthenticatewithprefix+ArangoDBusername+suffixagainsttheldapserverandsearchesfortherolesintheattributegroupMembership.

dn:uid=fermi,dc=example,dc=com

LDAPOptions

groupMembership:project-a

groupMembership:project-b

Thiswillgivethecombinedpermissionsoftherolesproject-aandproject-btotheuser.

Rolestransformationsandfilters

--ldap.roles-includecanbeusedtospecifyaregularexpressionthatisusedtofilterroles.Onlyrolesthatmatchtheregularexpressionareused.

--ldap.roles-excludecanbeusedtospecifyaregularexpressionthatisusedtofilterroles.Onlyrolesthatdonotmatchtheregularexpressionareused.

--ldap.roles-transformationcanbeusedtosepcifyaregularexpressionandreplacementtextas/re/text/.Thisregularexpressionisappliedtotherolenamefound.

--ldap.superuser-rolecanbeusedtospecifytheroleassociatedwiththesuperuser.Anyuserbelongingtothisrolegainssuperuserstatus.Thisroleischeckedbeforeapplyinganyregularexpression.

Example:

--ldap.prefixuid=\

--ldap.roles-include"^arangodb"

willonlyconsiderrolesthatstartwitharangodb.

--ldap.prefixuid=\

--ldap.roles-exclude"disabled"

willonlyconsiderrolesthatdocontaintheworddisabled.

--ldap.prefixuid=\

--ldap.superuser-role"arangodb-admin"

anyonebelongingtothegroup"arangodb-admin"willbecomeasuperuser.

bind+search

Examplewithanonymousauth:

--ldap.basedndc=company,dc=com\

--ldap.permissions-attribute-namearangodbPermissions

WiththisconfigurationArangoDBbindsanonymouslytotheLDAPserverandsearchesfortheuser.IftheuserisfoundaauthenticationisdonewiththeusersDNandpasswordandthendatabasepermissionsarefetched.

ExamplewithDNandpassword:

--ldap.binddncn=admin,dc=company,dc=com\

--ldap.bindpasswdadmin\

--ldap.permissions-attribute-namearangodbPermissions

LDAPOptions

WiththisconfigurationArangoDBbindswith--ldap.bindnand--ldap.bindpasswdtotheLDAPserverandsearchesfortheuser.IftheuserisfoundaauthenticationisdonewiththeusersDNandpasswordandthendatabasepermissionsarefetched.

Rolessearch

--ldap.roles-searchsearch-expression

Insteadofspecifyingarolesattributeitispossibletouseasearchwhenusingbind+search.Inthiscasethesearch-expressionmustbeanldapsearchstring.Any{USER}isreplacedbythednoftheuser.

Example:

--ldap.binddncn=admin,dc=company,dc=com\

--ldap.bindpasswdadmin\

--ldap.roles-search'(&(objectClass=groupOfUniqueNames)(uniqueMember={USER}))'

Additionaloptions

--ldap.search-filter"objectClass=*"

Restrictthesearchtospecificobjectclasses.ThedefaultisobjectClass=*.

--ldap.search-attribute"uid"

--ldap.search-attributespecifieswhichattributetocomparewiththeusername.Thedefaultisuid.

--ldap.search-scopesub

`--ldap.search-scopespecifiesinwhichscopetosearchforauser.Validareoneofbase,oneorsub.Thedefaultissub.

ldapurl

--ldap.urlldap://ldap.server.com:1234/dc=example,dc=com?uid?sub

Theldapurlconsistsoftheldapserverandport,abasedn,asearchattributeandascopewhichcanbeoneofbase,oneorsub.

TLSoptions

Aencryptedconnectioncanbeestablishedwith--ldap.tlstrueunderUNIXandGNU/Linuxplatforms.

AllfollowingoptionsarenotavailableunderWindows.

--ldap.tls

Thedefaultisfalse.Withtrueatlsconnectionisestablished.

--ldap.tls-version

Thedefaultis1.2.Availableversionsare1.0,1.1and1.2.

--ldap.tls-cert-check-strategy

Thedefaultishard.Availablestrategiesarenever,hard,demand,allowandtry.

--ldap.tls-cacert-file

LDAPOptions

Afilepathtooneormore(concatenated)certificateauthoritycertificatesinpemformat.Asdefaultnofilepathisconfigured.

Followingoptionhasnoeffect/doesnotworkundermacOS.

--ldap.tls-cacert-dir

Adirectorypathtocertificateauthoritycertificatesinc_rehashformat.Asdefaultnodirectorypathisconfigured.

LDAPOptions

Command-LineOptionsforLogging

Loglevelsandtopics

ArangoDB'slogoutputisgroupedintotopics.--log.levelcanbespecifiedmultipletimesatstartup,forasmanytopicsasneeded.Thelogverbosityandoutputfilescanbeadjustedperlogtopic.Forexample

--log.levelstartup=trace--log.levelqueries=trace--log.levelinfo

willlogmessagesconcerningstartupattracelevel,AQLqueriesattracelevelandeverythingelseatinfolevel.

Inaconfigurationfile,itiswrittenlikethis:

level=startup=trace

level=queries=trace

level=info

Notethattheremustnotbeanywhitespacearoundthesecond=.

Theavailableloglevelsare:

fatal:onlylogsfatalerrorserror:onlylogserrorswarning:onlylogswarningsanderrorsinfo:logsinformationmessages,warningsanderrorsdebug:logsdebugandinformationmessages,warningsanderrorstrace:logstrace,debugandinformationmessages,warningsanderrors

Notethatlevelsdebugandtracewillbeveryverbose.

SomerelevantlogtopicsavailableinArangoDB3are:

agency:informationabouttheagencycollector:informationabouttheWALcollector'sstatecompactor:informationaboutthecollectiondatafilecompactordatafiles:datafile-relatedoperationsmmap:informationaboutmemory-mappingoperations(includingmsync)performance:performance-releatedmessagesqueries:executedAQLqueries,slowqueriesreplication:replication-relatedinforequests:HTTPrequestsstartup:informationaboutserverstartupandshutdownthreads:informationaboutthreads

Logoutputs

Thelogoption--log.output<definition>allowsdirectingtheglobalorper-topiclogoutputtodifferentoutputs.Theoutputdefinition<definition>canbeoneof

-forstdin+forstderrsyslog://<syslog-facility>

syslog://<syslog-facility>/<application-name>

file://<relative-path>

Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputfordifferentlogtopics.Tosetupaper-topicoutputconfiguration,use--log.output<topic>=<definition>,e.g.

LoggingOptions

queries=file://queries.txt

logsallqueriestothefile"queries.txt".

Theoldoption--log.fileisstillavailablein3.0forconveniencereasons.In3.0itisashortcutforthemoregeneraloption--log.outputfile://filename.

Theoldoption--log.requests-fileisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption--log.outputrequests=file://....

Using--log.outputalsoallowsdirectinglogoutputtodifferentfilesbasedontopics.Forexample,tologallAQLqueriestoafile"queries.log"onecanusetheoptions:

--log.levelqueries=trace--log.outputqueries=file:///path/to/queries.log

ToadditionallylogHTTPrequesttoafilenamed"requests.log"addtheoptions:

--log.levelrequests=info--log.outputrequests=file:///path/to/requests.log

Forcingdirectoutput

Theoption--log.force-directcanbeusedtodisablelogginginanextraloggingthread.Ifsettotrue,anylogmessagesareimmediatelyprintedinthethreadthattriggeredthelogmessage.Thisisnon-optimalforperformancebutcanaiddebugging.Ifsettofalse,logmessagesarehandedofftoanextraloggingthread,whichasynchronouslywritesthelogmessages.

Localtime

Logdatesandtimesinlocaltimezone:--log.use-local-time

Ifspecified,alldatesandtimesinlogmessageswillusetheserver'slocaltime-zone.Ifnotspecified,alldatesandtimesinlogmessageswillbeprintedinUTC/Zulutime.ThedateandtimeformatusedinlogsisalwaysYYYY-MM-DDHH:MM:SS,regardlessofthissetting.IfUTCtimeisused,aZwillbeappendedtoindicateZulutime.

Colorlogging--log.colorvalue

Loggingtoterminaloutputisbydefaultcolored.Colorfulloggingcanbeturnedoffbysettingthevaluetofalse.

SourcefileandLinenumber

Loglinenumber:--log.line-number

Normally,ifanhumanreadablefatal,error,warningorinfomessageislogged,noinformationaboutthefileandlinenumberisprovided.Thefileandlinenumberisonlyloggedfordebugandtracemessage.Thisoptioncanbeusetoalwayslogthesepiecesofinformation.

Prefix

Logprefix:--log.prefixprefix

Thisoptionisusedspecifyanprefixtologgedtext.

Threads

Logthreadidentifier:--log.threadtrue

Wheneverlogoutputisgenerated,theprocessIDiswrittenaspartoftheloginformation.Settingthisoptionappendsthethreadidofthecallingthreadtotheprocessid.Forexample,

2010-09-20T13:04:01Z[19355]INFOreadyforbusiness

LoggingOptions

whennothreadisloggedand

2010-09-20T13:04:17Z[19371-18446744072487317056]readyforbusiness

whenthiscommandlineoptionisset.

Toalsologthreadnames,itispossibletosetthe--log.thread-nameoption.Bydefault--log.thread-nameissettofalse.

Logrole:--log.roletrue

Whensettotrue,thisoptionwillmaketheArangoDBloggerprintasinglecharacterwiththeserver'sroleintoeachloggedmessage.Therolesare:

U:undefined/unclear(usedatstartup)S:singleserverC:coordinatorP:primaryA:agent

Thedefaultvalueforthisoptionisfalse,sonoroleswillbelogged.

LoggingOptions

GeneralOptions

DatabaseUpgrade

--database.auto-upgrade

Specifyingthisoptionwillmaketheserverperformadatabaseupgradeatstart.AdatabaseupgradewillfirstcomparetheversionnumberstoredinthefileVERSIONinthedatabasedirectorywiththecurrentserverversion.

Ifthetwoversionnumbersmatch,theserverwillstartnormally.

Iftheversionnumberfoundinthedatabasedirectoryishigherthantheversionnumbertheserverisrunning,theserverexpectsthisisanunintentionaldowngradeandwillwarnaboutthis.Itwillhoweverstartnormally.Usingtheserverintheseconditionsishowevernotrecommendednorsupported.

Iftheversionnumberfoundinthedatabasedirectoryislowerthantheversionnumbertheserverisrunning,theserverwillcheckwhetherthereareanyupgradetaskstoperform.Itwillthenexecuteallrequiredupgradetasksandprinttheirstatuses.Ifoneoftheupgradetasksfails,theserverwillexitandrefusetostart.Re-startingtheserverwiththeupgradeoptionwillthenagaintriggertheupgradecheckandexecutionuntiltheproblemisfixed.Ifalltasksarefinished,theserverwillstartnormally.

Whetherornotthisoptionisspecified,theserverwillalwaysperformaversioncheckonstartup.Runningtheserverwithanon-matchingversionnumberintheVERSIONfilewillmaketheserverrefusetostart.

StorageEngine

AsofArangoDB3.2twostorageenginesaresupported.The"traditional"engineiscalledMMFiles,whichisalsothedefaultstorageengine.

AnalternativeenginebasedonRocksDBisalsoprovidedandcanbeturnedonmanually.

Onestorageenginetypeissupportedperserverperinstallation.Liveswitchingofstorageenginesonalreadyinstalledsystemsisn'tsupported.Configuringthewrongengine(notmatchingthepreviouslyusedone)willresultintheserverrefusingtostart.YoumayhoweveruseautotoletArangoDBchoosethepreviouslyusedone.

--server.storage-engine[auto|mmfiles|rocksdb]

Daemon

--daemon

Runstheserverasadaemon(asabackgroundprocess).Thisparametercanonlybesetifthepid(processid)fileisspecified.Thatis,unlessavaluetotheparameterpid-fileisgiven,thentheserverwillreportanerrorandexit.

DefaultLanguage

--default-languagedefault-language

Thedefaultlanguageistusedforsortingandcomparingstrings.Thelanguagevalueisatwo-letterlanguagecode(ISO-639)oritiscomposedbyatwo-letterlanguagecodewithandatwolettercountrycode(ISO-3166).Validlanguagesare"de","en","en_US"or"en_UK".

Thedefaultdefault-languageissettobethesystemlocaleonthatplatform.

Supervisor--supervisor

Executestheserverinsupervisormode.Intheeventthattheserverunexpectedlyterminatesduetoaninternalerror,thesupervisorwillautomaticallyrestarttheserver.Settingthisflagautomaticallyimpliesthattheserverwillrunasadaemon.Notethat,aswiththedaemonflag,thisflagrequiresthatthepid-fileparameterwillset.

unix>./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/

2012-06-27T15:58:28Z[10133]INFOstartingupinsupervisormode

GeneralOptions

Ascanbeseen(e.g.byexecutingthepscommand),thiswillstartasupervisorprocessandtheactualdatabaseprocess:

unix>psfax|greparangod

10137?Ssl0:00./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/

10142?Sl0:00\_./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/

Whenthedatabaseprocessterminatesunexpectedly,thesupervisorprocesswillstartupanewdatabaseprocess:

>kill-SIGSEGV10142

>psfax|greparangod

10137?Ssl0:00./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/

10168?Sl0:00\_./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/

Useridentity

--uiduid

Thename(identity)oftheusertheserverwillrunas.Ifthisparameterisnotspecified,theserverwillnotattempttochangeitsUID,sothattheUIDusedbytheserverwillbethesameastheUIDoftheuserwhostartedtheserver.Ifthisparameterisspecified,thentheserverwillchangeitsUIDafteropeningportsandreadingconfigurationfiles,butbeforeacceptingconnectionsoropeningotherfiles(suchasrecoveryfiles).Thisisusefulwhentheservermustbestartedwithraisedprivileges(incertainenvironments)butsecurityconsiderationsrequirethattheseprivilegesbedroppedoncetheserverhasstartedwork.

Observethatthisparametercannotbeusedtobypassoperatingsystemsecurity.Ingeneral,thisparameter(anditscorrespondingrelativegid)canlowerprivilegesbutnotraisethem.

Groupidentity

--gidgid

Thename(identity)ofthegrouptheserverwillrunas.Ifthisparameterisnotspecified,thentheserverwillnotattempttochangeitsGID,sothattheGIDtheserverrunsaswillbetheprimarygroupoftheuserwhostartedtheserver.Ifthisparameterisspecified,thentheserverwillchangeitsGIDafteropeningportsandreadingconfigurationfiles,butbeforeacceptingconnectionsoropeningotherfiles(suchasrecoveryfiles).

Thisparameterisrelatedtotheparameteruid.

Processidentity

--pid-filefilename

ThenameoftheprocessIDfiletousewhenrunningtheserverasadaemon.Thisparametermustbespecifiedifeithertheflagdaemonorsupervisorisset.

Checkmaxmemorymappings

--server.check-max-memory-mappingscanbeusedonLinuxtomakearangodcheckthenumberofmemorymappingscurrentlyusedbytheprocess(asreportedin/proc//maps)andcompareitwiththemaximumnumberofallowedmappingsasdeterminedby/proc/sys/vm/max_map_count.Ifthecurrentnumberofmemorymappingsgetsnearthemaximumallowedvalue,arangodwilllogawarninganddisallowthecreationoffurtherV8contextstemporarilyuntilthecurrentnumberofmappingsgoesdownagain.

Iftheoptionissettofalse,nosuchcheckswillbeperformed.Allnon-Linuxoperatingsystemsdonotprovidethisoptionandwillignoreit.

Console

--console

Runstheserverinanexclusiveemergencyconsolemode.Whenstartingtheserverwiththisoption,theserverisstartedwithaninteractiveJavaScriptemergencyconsole,withallnetworkingandHTTPinterfacesoftheserverdisabled.

GeneralOptions

Norequestscanbemadetotheserverinthismode,andtheonlywaytoworkwiththeserverinthismodeisbyusingtheemergencyconsole.Notethattheservercannotbestartedinthismodeifitisalreadyrunninginthisoranothermode.

RandomGenerator

--random.generatorarg

Theargumentisaninteger(1,2,3or4)whichsetsthemannerinwhichrandomnumbersaregenerated.Thedefaultmethod(3)istousetheanon-blockingrandom(orpseudorandom)numbergeneratorsuppliedbytheoperatingsystem.

Specifyinganargumentof2,usesablockingrandom(orpseudorandom)numbergenerator.Specifyinganargument1setsapseudorandomnumbergeneratorusinganimplicationoftheMersenneTwisterMT19937algorithm.Algorithm4isacombinationoftheblockingrandomnumbergeneratorandtheMersenneTwister.

Enable/disableauthentication

--server.authenticationSettingthisoptiontofalsewillturnoffauthenticationontheserversidesoallclientscanexecuteanyactionwithoutauthorizationandprivilegechecks.Thedefaultvalueistrue.

JWTSecret--server.jwt-secretsecret

ArangoDBwilluseJWTstoauthenticaterequests.UsingthisoptionletsyouspecifyaJWT.

InsingleserversetupsandwhennotspecifyingthissecretArangoDBwillgenerateasecret.

Inclusterdeploymentswhichhaveauthenticationenabledasecretmustbesetconsistentlyacrossallclustertaskssotheycantalktoeachother.

Enable/disableauthenticationforUNIXdomainsockets

--server.authentication-unix-socketsvalue

SettingvaluetotruewillturnoffauthenticationontheserversideforrequestscominginviaUNIXdomainsockets.Withthisflagenabled,clientslocatedonthesamehostastheArangoDBservercanuseUNIXdomainsocketstoconnecttotheserverwithoutauthentication.Requestscominginbyothermeans(e.g.TCP/IP)arenotaffectedbythisoption.

Thedefaultvalueisfalse.

Note:thisoptionisonlyavailableonplatformsthatsupportUNIXdomainsockets.

Enable/disableauthenticationforsystemAPIrequestsonly

--server.authentication-system-onlybooleanControlswhetherincomingrequestsneedauthenticationonlyiftheyaredirectedtotheArangoDB'sinternalAPIsandfeatures,locatedat/_api/,/_admin/etc.Iftheflagissettotrue,thenHTTPauthenticationisonlyrequiredforrequestsgoingtoURLsstartingwith/_,butnotforotherURLs.Theflagcanthusbeusedtoexposeauser-madeAPIwithoutHTTPauthenticationtotheoutsideworld,buttopreventtheoutsideworldfromusingtheArangoDBAPIandtheadmininterfacewithoutauthentication.NotethatcheckingtheURLisperformedafteranydatabasenameprefixhasbeenremoved.ThatmeanswhentheactualURLcalledis/_db/_system/myapp/myaction,theURL/myapp/myactionwillbeusedforauthentication-system-onlycheck.Thedefaultistrue.NotethatauthenticationstillneedstobeenabledfortheserverregularlyinorderforHTTPauthenticationtobeforcedfortheArangoDBAPIandthewebinterface.Settingonlythisflagisnotenough.YoucancontrolArangoDB'sgeneralauthenticationfeaturewiththe--server.authenticationflag.

Enableauthenticationcachetimeout

--server.authentication-timeoutvalue

Setsthecachetimeouttovalue(inseconds).ThisisonlynecessaryifyouuseanexternalauthenticationsystemlikeLDAP.

Enablelocalauthentication--server.local-authenticationvalue

GeneralOptions

Ifsettofalseonlyusetheexternalauthenticationsystem.Iftruealsousethelocal_userscollections.

Thedefaultvalueistrue.

Enable/disablereplicationapplier

--database.replication-applierflag

Iffalsetheserverwillstartwithreplicationappliersturnedoff,evenifthereplicationappliersareconfiguredwiththeautoStartoption.Usingthecommand-lineoptionwillnotchangethevalueoftheautoStartoptionintheapplierconfiguration,butwillsuppressauto-startingthereplicationapplierjustonce.

Iftheoptionisnotused,ArangoDBwillreadtheapplierconfigurationfromthefileREPLICATION-APPLIER-CONFIGonstartup,andusethevalueoftheautoStartattributefromthisfile.

Thedefaultistrue.

Keep-alivetimeout--http.keep-alive-timeout

AllowstospecifythetimeoutforHTTPkeep-aliveconnections.Thetimeoutvaluemustbespecifiedinseconds.Idlekeep-aliveconnectionswillbeclosedbytheserverautomaticallywhenthetimeoutisreached.Akeep-alive-timeoutvalue0willdisablethekeepalivefeatureentirely.

HideProductheader

--http.hide-product-header

Iftrue,theserverwillexcludetheHTTPheader"Server:ArangoDB"inHTTPresponses.Ifsettofalse,theserverwillsendtheheaderinresponses.

Thedefaultisfalse.

Allowmethodoverride--http.allow-method-override

Whenthisoptionissettotrue,theHTTPrequestmethodwilloptionallybefetchedfromoneofthefollowingHTTPrequestheadersifpresentintherequest:

x-http-methodx-http-method-overridex-method-override

Iftheoptionissettotrueandanyoftheseheadersisset,therequestmethodwillbeoverriddenbythevalueoftheheader.Forexample,thisallowsissuinganHTTPDELETErequestwhichtotheoutsideworldwilllooklikeanHTTPGETrequest.Thisallowsbypassingproxiesandtoolsthatwillonlyletcertainrequesttypespass.

Settingthisoptiontotruemayimposeasecurityrisksoitshouldonlybeusedincontrolledenvironments.

Thedefaultvalueforthisoptionisfalse.

Serverthreads

--server.threadsnumber

Specifiesthenumberofthreadsthatarespawnedtohandlerequests.

Togglingserverstatistics

--server.statisticsvalue

Ifthisoptionisvalueisfalse,thenArangoDB'sstatisticsgatheringisturnedoff.StatisticsgatheringcausesregularCPUactivitysousingthisoptiontoturnitoffmightrelieveheavy-loadedinstancesabit.

GeneralOptions

Sessiontimeout

timetoliveforserversessions--server.session-timeoutvalue

Thetimeoutforwebinterfacesessions,usingforauthenticatingrequeststothewebinterface(/_admin/aardvark)andrelatedareas.

Sessionsareonlyusedwhenauthenticationisturnedon.

Foxxqueues

enableordisabletheFoxxqueuesfeature--foxx.queuesflagIftrue,theFoxxqueueswillbeavailableandjobsinthequeueswillbeexecutedasynchronously.Thedefaultistrue.Whensettofalsethequeuemanagerwillbedisabledandanyjobsarepreventedfrombeingprocessed,whichmayreduceCPUloadabit.

Foxxqueuespollinterval

pollintervalforFoxxqueues--foxx.queues-poll-intervalvalueThepollintervalfortheFoxxqueuesmanager.Thevalueisspecifiedinseconds.LowervalueswillmeanmoreimmediateandmorefrequentFoxxqueuejobexecution,butwillmakethequeuethreadwakeupandquerythequeuesmoreoften.Whensettoalowvalue,thequeuethreadmightcauseCPUload.Thedefaultis1second.IfFoxxqueuesarenotusedmuch,thenthisvaluemaybeincreasedtomakethequeuesthreadwakeupless.

Directory

TheWALlogfilesdirectory:--wal.directory

Specifiesthedirectoryinwhichthewrite-aheadlogfilesshouldbestored.Ifthisoptionisnotspecified,itdefaultstothesubdirectoryjournalsintheserver'sglobaldatabasedirectory.Ifthedirectoryisnotpresent,itwillbecreated.

Logfilesize

thesizeofeachWALlogfile--wal.logfile-sizeSpecifiesthefilesize(inbytes)foreachwrite-aheadlogfile.Thelogfilesizeshouldbechosensothateachlogfilecanstoreaconsiderableamountofdocuments.Thebiggerthelogfilesizeischosen,thelongeritwilltaketofillupasinglelogfile,whichalsoinfluencesthedelayuntilthedatainalogfilewillbegarbage-collectedandwrittentocollectionjournalsanddatafiles.Italsoaffectshowlonglogfilerecoverywilltakeatserverstart.

Allowoversizeentries

whetherornotoversizeentriesareallowed--wal.allow-oversize-entriesWhetherornotitisallowedtostoreindividualdocumentsthatarebiggerthanwouldfitintoasinglelogfile.Settingtheoptiontofalsewillmakesuchoperationsfailwithanerror.Settingtheoptiontotruewillmakesuchoperationssucceed,butwithahighpotentialperformanceimpact.Thereasonisthatforeachoversizeoperation,anindividualoversizelogfileneedstobecreatedwhichmayalsoblockotheroperations.Theoptionshouldbesettofalseifitiscertainthatdocumentswillalwayshaveasizesmallerthanasinglelogfile.

Numberofreservelogfiles

maximumnumberofreservelogfiles--wal.reserve-logfilesThemaximumnumberofreservelogfilesthatArangoDBwillcreateinabackgroundprocess.Reservelogfilesareusefulinthesituationwhenanoperationneedstobewrittentoalogfilebutthereservespaceinthelogfileistoolowforstoringtheoperation.Inthiscase,anewlogfileneedstobecreatedtostoretheoperation.Creatingnewlogfilesisnormallyslow,soArangoDBwilltrytopre-createlogfilesinabackgroundprocesssotherearealwaysreservelogfileswhentheactivelogfilegetsfull.ThenumberofreservelogfilesthatArangoDBkeepsinthebackgroundisconfigurablewiththisoption.

Numberofhistoriclogfiles

maximumnumberofhistoriclogfiles--wal.historic-logfilesThemaximumnumberofhistoriclogfilesthatArangoDBwillkeepaftertheyhavebeengarbage-collected.Ifnoreplicationisused,thereisnoneedtokeephistoriclogfilesexceptforhavingalocalchangelog.Inareplicationsetup,thenumberofhistoriclogfilesaffectstheamountofdataaslavecanfetchfromthemaster'slogs.Themorehistoriclogfiles,themorehistoricdataisavailableforaslave,whichisusefuliftheconnectionbetweenmasterandslaveisunstableorslow.Nothavingenoughhistoriclogfilesavailablemightleadtologfiledatabeingdeletedonthemasteralreadybeforeaslavehasfetchedit.

Syncinterval

intervalforautomatic,non-requesteddisksyncs--wal.sync-intervalTheinterval(inmilliseconds)thatArangoDBwillusetoautomaticallysynchronizedatainitswrite-aheadlogstodisk.Automaticsyncswillonlybeperformedfornot-yetsynchronizeddata,andonlyforoperationsthathavebeenexecutedwithoutthewaitForSyncattribute.

Flushtimeout

WALflushtimeout`--wal.flush-timeoutThetimeout(inmilliseconds)thatArangoDBwillatmostwaitwhenflushingafullWALlogfiletodisk.Whenthetimeoutisreachedandtheflushisnotcompleted,theoperationthatrequestedtheflushwillfailwithalocktimeouterror.

Throttling

ThrottlewritestoWALwhenatleastsuchmanyoperationsarewaitingforgarbagecollection:--wal.throttle-when-pending

Themaximumvalueforthenumberofwrite-aheadloggarbage-collectionqueueelements.Ifsetto0,thequeuesizeisunbounded,andnowrite-throttlingwilloccur.Ifsettoanon-zerovalue,write-throttlingwillautomaticallykickinwhenthegarbage-collectionqueuecontainsatleastasmanyelementsasspecifiedbythisoption.Whilewrite-throttlingisactive,data-modificationoperationswillintentionallybedelayedbyaconfigurableamountoftime.Thisistoensurethewrite-aheadloggarbagecollectorcancatchupwiththeoperationsexecuted.Write-throttlingwillstayactiveuntilthegarbage-collectionqueuesizegoesdownbelowthespecifiedvalue.Write-throttlingisturnedoffbydefault.

--wal.throttle-wait

Thisoptiondeterminesthemaximumwaittime(inmilliseconds)foroperationsthatarewrite-throttled.Ifwrite-throttlingisactiveandanewwriteoperationistobeexecuted,itwillwaitforatmostthespecifiedamountoftimeforthewrite-aheadloggarbage-collectionqueuesizetofallbelowthethrottlingthreshold.Ifthequeuesizedecreasesbeforethemaximumwaittimeisover,theoperationwillbeexecutednormally.Ifthequeuesizedoesnotdecreasebeforethewaittimeisover,theoperationwillbeabortedwithanerror.Thisoptiononlyhasaneffectif--wal.throttle-when-pendinghasanon-zerovalue,whichisnotthedefault.

Numberofslots

Maximumnumberofslotstobeusedinparallel:--wal.slots

Configurestheamountofwriteslotsthewrite-aheadlogcangivetowriteoperationsinparallel.Anywriteoperationwillleaseaslotandreturnittothewrite-aheadlogwhenitisfinishedwritingthedata.Aslotwillremainblockeduntilthedatainitwassynchronizedtodisk.Afterthat,aslotbecomesreusablebyfollowingoperations.Therequirednumberofslotsisthusdeterminedbytheparallelityofwriteoperationsandthedisksynchronizationspeed.Slowdisksprobablyneedhighervalues,andfastdisksmayonlyrequireavaluelowerthanthedefault.

Ignorelogfileerrors

Ignorelogfileerrorswhenopeninglogfiles:--wal.ignore-logfile-errors

Ignoresanyrecoveryerrorscausedbycorruptedlogfilesonstartup.Whensettofalse,therecoveryprocedureonstartupwillfailwithanerrorwheneveritencountersacorrupted(thatincludesonlyhalf-written)logfile.Thisisasecurityprecautiontopreventdatalossincaseofdiskerrorsetc.Whentherecoveryprocedureabortsbecauseofcorruption,anycorruptedfilescanbeinspectedandfixed(orremoved)manuallyandtheservercanberestartedafterwards.

Settingtheoptiontotruewillmaketheservercontinuewiththerecoveryprocedureevenincaseitdetectscorruptlogfileentries.Inthiscaseitwillstopatthefirstcorruptedlogfileentryandignoreallothers,whichmightcausedataloss.

Ignorerecoveryerrors

Ignorerecoveryerrors:--wal.ignore-recovery-errors

Ignoresanyrecoveryerrorsnotcausedbycorruptedlogfilesbutbylogicalerrors.Logicalerrorscanoccuriflogfilesoranyotherserverdatafileshavebeenmanuallyeditedortheserverissomehowmisconfigured.

Ignore(non-WAL)datafileerrors

Ignoredatafileerrorswhenloadingcollections:--database.ignore-datafile-errorsboolean

Ifsettofalse,CRCmismatchandothererrorsincollectiondatafileswillleadtoacollectionnotbeingloadedatall.Thecollectioninthiscasebecomesunavailable.IfsuchcollectionneedstobeloadedduringWALrecovery,theWALrecoverywillalsoabort(ifnotforcedwithoption--wal.ignore-recovery-errorstrue).

Settingthisflagtofalseprotectsusersfromunintentionallyusingacollectionwithcorrupteddatafiles,fromwhichonlyasubsetoftheoriginaldatacanberecovered.Workingwithsuchcollectioncouldleadtodatalossandfollowuperrors.Inordertoaccesssuchcollection,itisrequiredtoinspectandrepairthecollectiondatafilewiththedatafiledebugger(arango-dfdb).

Ifsettotrue,CRCmismatchandothererrorsduringtheloadingofacollectionwillleadtothedatafilebeingpartiallyloaded,uptothepositionofthefirsterror.Alldatauptountiltheinvalidpositionwillbeloaded.Thiswillenableuserstocontinuewithcollectiondatafileseveniftheyarecorrupted,butthiswillresultinonlyapartialloadoftheoriginaldataandpotentialfollowuperrors.TheWALrecoverywillstillabortwhenencounteringacollectionwithacorrupteddatafile,atleastif--wal.ignore-recovery-errorsisnotsettotrue.

Thedefaultvalueisfalse,socollectionswithcorrupteddatafileswillnotbeloadedatall,preventingpartialloadsandfollowuperrors.However,ifsuchcollectionisrequiredatserverstartup,duringWALrecovery,theserverwillaborttherecoveryandrefusetostart.

MMFilesCompactionoptionsTheArangoDBMMFilesstorageenginewillrunacompactionoverdatafiles.

ArangoDBwritesDocumentsintheWALfile.Oncetheyhavebeensealedinthewalfile,thecollectormaycopythemintoapercollectionjournalfile.

Oncejournalfilesfillup,they'resealedtobecomedatafiles.

=>onecollectionmayhavedocumentsintheWALlogs,itsjournalfile,andanarbitrarynumberofdatafiles.

Ifacollectionisloaded,eachofthesefilesareopened(thususeafilehandle)andaremmap'ed.Sincefilehandlesandmemorymappedfilesarealsoasparseresource,thatnumbershouldbekeptlow.

Onceyouupdateorremovedocumentsfromdatafiles(oralreadydidwhileitwasthejournalfile)thesedocumentsaremarkedas'dead'withadeletionmarker.

Overtimethenumberofdeaddocumentsmayrise,andwedon'twanttousethepreviouslymentionedresources,plusthediskspaceshouldbegivenbacktothesystem.Thusseveraljournalfilescanbecombinedtoone,ommittingthedeaddocuments.

Combiningseveralofthesedatafilesintooneiscalledcompaction.Thecompactionprocessreadsthealivedocumentsfromtheoriginaldatafiles,andwritesthemintonewdatafile.

Oncethatisdone,thememorymappingstotheolddatafilesisreleased,andthefilesareerased.

Sincethecompactionlocksthecollection,andalsousesI/Oresources,itscarefullyconfigurableunderwhichconditionsthesystemshouldperformwhichamountofthesecompactionjobs:

ArangoDBspawnsonecompactorthreadperdatabase.Thesettingsbelowvaryinscope.

ActivitycontrolTheactivitycontrolparametersalterthebehaviourintermsofscan/executionfrequencyofthecompaction.

Sleepintervalbetweentwocompactionruns(inseconds):--compaction.db-sleep-timeThenumberofsecondsthecollectorthreadwillwaitbetweentwoattemptstosearchforcompactabledatafilesofcollectionsinoneDatabase.Ifthecompactorhasactuallyexecutedwork,asubsequentlookupisdone.Scope:Database.

Minimumsleeptimebetweentwocompactionruns(inseconds):--compaction.min-intervalWhenanactualcompactionwasexecutedforonecollection,wewaitforthistimebeforeweexecutethecompactiononthiscollectionagain.Thisisheretoleteventuallypiledupuserloadbeworkedout.Scope:collection.

SourcedatafilesTheseparameterscontrolwhichdatafilesaretakenintoaccountforacompactionrun.Youcanspecifyseveralcriteriawhicheachoffmaybesufficcientalone.

Thescanoverthedatafilesbelongingtoonecollectionisexecutedfromoldestdatafiletonewest;iffilesqualifyforacompactiontheymaybemergedwithnewerfiles(containingyoungerdocuments)

Scope:Collectionlevel,someareinfluencedbycollectionsettings.

minimalfilesizethresholdoriginaldatafileshavetobebelowforacompaction:--compaction.min-small-data-file-sizeThisisthethresholdwhichcontrolsbelowwhichminimumtotalsizeadatafilewillalwaysbetakenintoaccountforthecompaction.

Minimumunusedcountofdocumentsinadatafile:--compaction.dead-documents-thresholdDatafileswilloftencontaindeaddocuments.Thisparameterspecifiestheirtopmostaccetpeablecountuntilthedatafilequalifiesforcompaction.

Howmanybytesofthesourcedatafileareallowedtobeunusedatmost:--compaction.dead-size-thresholdThedeaddatasizevariesalongwiththesizeofyourdocuments.Ifyouhavemanybigdocuments,thisthresholdmayhitbeforethedocumentcountthreshold.

CompactionOptions

Howmanypercentofthesourcedatafileshouldbeunusedatleast:--compaction.dead-size-percent-thresholdsincethesizeofthedocumentsmayvarythisthresholdworksonthepercentageofthedeaddocumentssize.Thus,ifyouhavemanyhugedeaddocuments,thisthresholdkicksinearlier.

Tonameanexamplewithnumbers,ifthedatafilecontains800kbytesofaliveand400kbytesofdeaddocuments,theshareofthedeaddocumentsis:

400/(400+800)=33%.

Ifthisvalueifhigherthanthespecifiedthreshold,thedatafilewillbecompacted.

CompactedtargetfilesOncedatafilesofacollectionarequalifiedforacompactionrun,theseparameterscontrolhowmanydatafilesaremergedintoone,(orevenonesourcedatafilemaybecompactedintoonesmallertargetdatafile)

Scope:Collectionlevel,someareinfluencedbycollectionsettings.

Maximumnumberoffilestomergetoonefile:--compaction.dest-max-filesHowmanydatafiles(atmost)wemaymergeintooneresultingdatafileduringonecompactionrun.

Howlargetheresultingfilemaybeincomparisontothecollectionsdatabase.maximal-journal-sizesetting:--compaction.dest-max-file-size-factorInArangoDByoucanconfigureadefaultjournalfilesizegloballyandoverrideitonapercollectionlevel.Thisvaluecontrolsthesizeofcollecteddatafilesrelativetotheconfiguredjournalfilesizeofthecollectioninquestion.

Afactorof3meansthatthemaximumfilesizeofthecompactedfileis3timesthesizeofthemaximumcollectionjournalfilesize.

howlargemaythecompactionresultfilebecome:--compaction.dest-max-result-file-sizenexttothefactorabove,atotallymaximumallowedfilesizeinbytesmaybespecified.Thiswilloverruleallpreviousparameters.

CompactionOptions

ClustersOptions

Agencyendpoint

Listofagencyendpoints:--cluster.agency-endpointendpoint

Anagencyendpointtheservercanconnectto.Theoptioncanbespecifiedmultipletimes,sotheservercanuseaclusterofagencyservers.Endpointshavethefollowingpattern:

tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryptionssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryption

AtleastoneendpointmustbespecifiedorArangoDBwillrefusetostart.ItisrecommendedtospecifyatleasttwoendpointssoArangoDBhasanalternativeendpointifoneofthembecomesunavailable.

Examples

--cluster.agency-endpointtcp://192.168.1.1:4001--cluster.agency-endpointtcp://192.168.1.2:4002...

Myaddress

Thisserver'saddress/endpoint:--cluster.my-addressendpoint

Theserver'sendpointforcluster-internalcommunication.Ifspecified,itmusthavethefollowingpattern:

tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryptionssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryption

Ifnoendpointisspecified,theserverwilllookupitsinternalendpointaddressintheagency.Ifnoendpointcanbefoundintheagencyfortheserver'sid,ArangoDBwillrefusetostart.

Examples

Listenonlyoninterfacewithaddress192.168.1.1

--cluster.my-addresstcp://192.168.1.1:8530

Listenonallipv4andipv6addresses,whichareconfiguredonport8530

--cluster.my-addressssl://[::]:8530

Myrole

Thisserver'srole:--cluster.my-role[dbserver|coordinator]

Theserver'srole.Isthisinstanceadbserver(backenddataserver)oracoordinator(frontendserverforexternalandapplicationaccess)

NodeID(deprecated)

Thisserver'sid:--cluster.my-local-infoinfo

Somelocalinformationabouttheserverinthecluster,thiscanforexamplebeanIPaddresswithaprocessIDoranystringuniquetotheserver.Specifyinginfoismandatoryonstartupiftheserverid(seebelow)isnotspecified.Eachserveroftheclustermusthaveauniquelocalinfo.Thisisignoredifmy-idbelowisspecified.

ClusterOptions

Thisoptionisdeprecatedandwillberemovedinafuturerelease.TheclusternodeidshavebeendroppedinfavourofoncegeneratedUUIDs.

Moreadvancedoptions(shouldgenerallyremainuntouched)

Synchroneousreplicationtiming:--cluster.synchronous-replication-timeout-factordouble

Strechorclinchtimeoutsforinternalsynchroneousreplicationmechanismbetweendbservers.Allsuchtimeoutsareaffectedbythischange.Pleasechangeonlywithintentandgreatcare.Defaultat1.0.

Systemreplicationfactor:--cluster.system-replication-factorinteger

Changedefaultreplicationfactorforsystemcollections.Defaultat2.

ClusterOptions

RocksDBengineoptionsRocksDBisahighlyconfigurablekey-valuestoreusedtopowerourRocksDBstorageengine.Mostoftheoptionsonthispagearepass-throughoptionstotheunderlyingRocksDBinstance,andwechangeveryfewoftheirdefaultsettings.

Dependingonthestorageengineyouhavechosentheavailabilityandthescopeoftheseoptionschanges.

Incaseyouhavechosenmmfilessomeofthefollowingoptionsapplytopersistentindexes.Incaseofrocksdbitwillapplytoalldatastoredaswellasindexes.

Pass-throughoptions--rocksdb.wal-directory

AbsolutepathfortheRocksDBWALfiles.Ifleftempty,thiswilluseasubdirectoryjournalsinsidethedatadirectory.

Writebuffers

--rocksdb.write-buffer-size

Theamountofdatatobuildupineachin-memorybuffer(backedbyalogfile)beforeclosingthebufferandqueuingittobeflushedintostandardstorage.Default:64MiB.Largervaluesmayimproveperformance,especiallyforbulkloads.

--rocksdb.max-write-buffer-number

Themaximumnumberofwritebuffersthatbuiltupinmemory.Ifthisnumberisreachedbeforethebufferscanbeflushed,writeswillbeslowedorstalled.Default:2.

--rocksdb.min-write-buffer-number-to-merge

Minimumnumberofwritebuffersthatwillbemergedtogetherwhenflushingtonormalstorage.Default:1.

--rocksdb.max-total-wal-size

MaximumtotalsizeofWALfilesthat,whenreached,willforceaflushofallcolumnfamilieswhosedataisbackedbytheoldestWALfiles.Settingthistoalowvaluewilltriggerregularflushingofcolumnfamilydatafrommemtables,sothatWALfilescanbemovedtothearchive.SettingthistoahighvaluewillavoidregularflushingbutmaypreventWALfilesfrombeingmovedtothearchiveandbeingremoved.

--rocksdb.delayed-write-rate(Hidden)

LimitedwriteratetoDB(inbytespersecond)ifwearewritingtothelastin-memorybufferallowedandweallowmorethan3buffers.Default:16MiB/s.

LSMtreestructure

--rocksdb.num-levels

ThenumberoflevelsforthedatabaseintheLSMtree.Default:7.

--rocksdb.num-uncompressed-levels

Thenumberoflevelsthatdonotusecompression.Thedefaultvalueis2.LevelsabovethisnumberwilluseSnappycompressiontoreducethediskspacerequirementsforstoringdataintheselevels.

--rocksdb.dynamic-level-bytes

Iftrue,theamountofdataineachleveloftheLSMtreeisdetermineddynamicallysoastominimizethespaceamplification;otherwise,thelevelsizesarefixed.ThedynamicsizingallowsRocksDBtomaintainawell-structuredLSMtreeregardlessoftotaldatasize.Default:true.

--rocksdb.max-bytes-for-level-base

Themaximumtotaldatasizeinbytesinlevel-1oftheLSMtree.Onlyeffectiveif--rocksdb.dynamic-level-bytesisfalse.Default:256MiB.

--rocksdb.max-bytes-for-level-multiplier

ThemaximumtotaldatasizeinbytesforlevelLoftheLSMtreecanbecalculatedasmax-bytes-for-level-base*(max-bytes-for-level-multiplier^(L-1)).Onlyeffectiveif--rocksdb.dynamic-level-bytesisfalse.Default:10.

--rocksdb.level0-compaction-trigger

Compactionoflevel-0tolevel-1istriggeredwhenthismanyfilesexistinlevel-0.Settingthistoahighernumbermayhelpbulkwritesattheexpenseofslowingdownreads.Default:2.

--rocksdb.level0-slowdown-trigger

Whenthismanyfilesaccumulateinlevel-0,writeswillbesloweddownto--rocksdb.delayed-write-ratetoallowcompactiontocatchup.Default:20.

--rocksdb.level0-stop-trigger

Whenthismanyfilesaccumulateinlevel-0,writeswillbestoppedtoallowcompactiontocatchup.Default:36.

FileI/O

--rocksdb.compaction-read-ahead-size

Ifnon-zero,weperformbiggerreadswhendoingcompaction.Ifyou'rerunningRocksDBonspinningdisks,youshouldsetthistoatleast2MiB.ThatwayRocksDB'scompactionisdoingsequentialinsteadofrandomreads.Default:0.

--rocksdb.use-direct-reads(Hidden)

OnlymeaningfulonLinux.Ifset,useO_DIRECTforreadingfiles.Default:false.

--rocksdb.use-direct-io-for-flush-and-compaction(Hidden)

OnlymeaningfulonLinux.Ifset,useO_DIRECTforwritingfiles.Default:false.

--rocksdb.use-fsync(Hidden)

Ifset,issueanfsynccallwhenwritingtodisk(settofalsetoissuefdatasynconly.Default:false.

Backgroundtasks

--rocksdb.max-background-jobs

Maximumnumberofconcurrentbackgroundcompactionjobs,submittedtothelowprioritythreadpool.Default:numberofprocessors.

--rocksdb.num-threads-priority-high

Numberofthreadsforhighpriorityoperations(e.g.flush).Werecommendsettingthisequaltomax-background-flushes.Default:numberofprocessors/2.

--rocksdb.num-threads-priority-low

Numberofthreadsforlowpriorityoperations(e.g.compaction).Default:numberofprocessors/2.

Caching

--rocksdb.block-cache-size

Thisisthesizeoftheblockcacheinbytes.Increasingthismayimproveperformance.Ifthereislessthan4GiBofRAMonthesystem,thedefaultvalueis256MiB.Ifthereismore,thedefaultis(systemRAMsize-2GiB)*0.3.

--rocksdb.block-cache-shard-bits

Thenumberofbitsusedtoshardtheblockcachetoallowconcurrentoperations.Tokeepindividualshardsatareasonablesize(i.e.atleast512KB),keepthisvaluetoatmostblock-cache-shard-bits/512KB.Default:block-cache-size/2^19.

--rocksdb.table-block-size

Approximatesizeofuserdata(inbytes)packedperblockforuncompresseddata.

--rocksdb.recycle-log-file-num(Hidden)

Numberoflogfilestokeeparoundforrecycling.Default:0.

Miscellaneous

--rocksdb.optimize-filters-for-hits(Hidden)

Thisflagspecifiesthattheimplementationshouldoptimizethefiltersmainlyforcaseswherekeysarefoundratherthanalsooptimizeforthecasewherekeysarenot.Thiswouldbeusedincaseswheretheapplicationknowsthatthereareveryfewmissesortheperformanceinthecaseofmissesisnotasimportant.Default:false.

--rocksdb.wal-recovery-skip-corrupted(Hidden)

Iftrue,skipcorruptedrecordsinWALrecovery.Default:false.

Non-Pass-ThroughOptions--rocksdb.wal-file-timeout(Hidden)

TimeoutafterwhichunusedWALfilesaredeleted(inseconds).Default:10.0s.

DataofongoingtransactionsisstoredinRAM.Transactionsthatgettoobig(intermsofnumberofoperationsinvolvedorthetotalsizeofdatacreatedormodifiedbythetransaction)willbecommittedautomatically.EffectivelythismeansthatbigusertransactionsaresplitintomultiplesmallerRocksDBtransactionsthatarecommittedindividually.TheentireusertransactionwillnotnecessarilyhaveACIDpropertiesinthiscase.

ThefollowingoptionscanbeusedtocontroltheRAMusageandautomaticintermediatecommitsfortheRocksDBengine:

--rocksdb.max-transaction-size

Transactionsizelimit(inbytes).TransactionsstoreallkeysandvaluesinRAM,solargetransactionsruntheriskofcausingout-of-memorysitations.Thissettingallowsyoutoensurethatdoesnothappenbylimitingthesizeofanyindividualtransaction.TransactionswhoseoperationswouldconsumemoreRAMthanthisthresholdvaluewillabortautomaticallywitherror32("resourcelimitexceeded").

--rocksdb.intermediate-commit-size

Ifthesizeofalloperationsinatransactionreachesthisthreshold,thetransactioniscommittedautomaticallyandanewtransactionisstarted.Thevalueisspecifiedinbytes.

--rocksdb.intermediate-commit-count

Ifthenumberofoperationsinatransactionreachesthisvalue,thetransactioniscommittedautomaticallyandanewtransactionisstarted.

--rocksdb.throttle

Ifenabled,throttlestheingestrateofwritesifnecessarytoreducechancesofcompactionsgettingtoofarbehindandblockingincomingwrites.Thisoptionistruebydefault.

HashcacheoptionsSinceArangoDB3.2,theseveralcorecomponentsoftheserveruseacachesystemwhichpoolsmemoryacrossmanydifferentcachetables.Inordertoprovideintelligentinternalmemorymanagement,thesystemperiodicallyreclaimsmemoryfromcacheswhichareusedlessoftenandreallocatesittocacheswhichgetmoreactivity.

Cachesize

Globalsizelimitforallhashcaches:--cache.size

Theglobalcachingsystem,allcaches,andallthedatacontainedthereinwillfitinsidethislimit.Thesizeisspecifiedinbytes.Ifthereislessthan4GiBofRAMonthesystem,thedefaultvalueis256MiB.Ifthereismore,thedefaultis(systemRAMsize-2GiB)*0.3.

Rebalancinginterval

Timebetweencacherebalancingattempts:--cache.rebalancing-interval

Thevalueisspecifiedinmicrosecondswithadefaultof2secondsandaminimumof500milliseconds.

HashCacheOptions

AsynchronousTasks

maximalqueuesize

Maximumsizeofthequeueforrequests:--server.maximal-queue-sizesize

Specifiesthemaximumsizeofthequeueforasynchronoustaskexecution.Ifthequeuealreadycontainssizetasks,newtaskswillberejecteduntilothertasksarepoppedfromthequeue.Settingthisvaluemayhelppreventingfromrunningoutofmemoryifthequeueisfilledupfasterthantheservercanprocessrequests.

AsynchronousTasks

DurabilityConfiguration

GlobalConfiguration

Thereareglobalconfigurationvaluesfordurability,whichcanbeadjustedbyspecifyingthefollowingconfigurationoptions:

defaultwaitforsyncbehavior--database.wait-for-syncbooleanDefaultwait-for-syncvalue.Canbeoverwrittenwhencreatinganewcollection.Thedefaultisfalse.

forcesyncingofcollectionpropertiestodisk--database.force-sync-propertiesbooleanForcesyncingofcollectionpropertiestodiskaftercreatingacollectionorupdatingitsproperties.Ifturnedoff,nofsyncwillhappenforthecollectionanddatabasepropertiesstoredinparameter.jsonfilesinthefilesystem.Turningoffthisoptionwillspeedupworkloadsthatcreateanddropalotofcollections(e.g.testsuites).Thedefaultistrue.

intervalforautomatic,non-requesteddisksyncs--wal.sync-intervalTheinterval(inmilliseconds)thatArangoDBwillusetoautomaticallysynchronizedatainitswrite-aheadlogstodisk.Automaticsyncswillonlybeperformedfornot-yetsynchronizeddata,andonlyforoperationsthathavebeenexecutedwithoutthewaitForSyncattribute.

Per-collectionconfiguration

Youcanalsoconfigurethedurabilitybehavioronaper-collectionbasis.UsetheArangoDBshelltochangetheseproperties.

getsorsetsthepropertiesofacollectioncollection.properties()Returnsanobjectcontainingallcollectionproperties.

waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheMMFilesstorageengineonly.isVolatile:IftruethenthecollectiondatawillbekeptinmemoryonlyandArangoDBwillnotwriteorsyncthedatatodisk.ThisoptionismeaningfulfortheMMFilesstorageengineonly.keyOptions(optional)additionaloptionsforkeygeneration.ThisisaJSONarraycontainingthefollowingattributes(note:someoftheattributesareoptional):

type:thetypeofthekeygeneratorusedforthecollection.allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeofdocumentsisconsideredanerror.increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.

indexBuckets:numberofbucketsintowhichindexesusingahashtablearesplit.Thedefaultis16andthisnumberhastobeapowerof2andlessthanorequalto1024.ThisoptionismeaningfulfortheMMFilesstorageengineonly.Forverylargecollectionsoneshouldincreasethistoavoidlongpauseswhenthehashtablehastobeinitiallybuiltorresized,sincebucketsareresizedindividuallyandcanbeinitiallybuiltinparallel.Forexample,64mightbeasensiblevalueforacollectionwith100000000documents.Currently,onlytheedgeindexrespectsthisvalue,butotherindextypesmightfollowinfutureArangoDBversions.Changes(seebelow)areappliedwhenthecollectionisloadedthenexttime.Inaclustersetup,theresultwillalsocontainthefollowingattributes:numberOfShards:thenumberofshardsofthecollection.shardKeys:containsthenamesofdocumentattributesthatareusedtodeterminethetargetshardfordocuments.replicationFactor:determineshowmanycopiesofeachshardarekeptondifferentDBServers.collection.properties(properties)Changesthecollectionproperties.propertiesmustbeanobjectwithoneormoreofthefollowingattribute(s):waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheMMFilesstorageengineonly.indexBuckets:Seeabove,changesareonlyappliedwhenthecollectionisloadedthenexttime.ThisoptionismeaningfulfortheMMFilesstorageengineonly.replicationFactor:ChangethenumberofshardcopieskeptondifferentDBServers,validvaluesareintegernumbersintherangeof1-10(Clusteronly)Note:itisnotpossibletochangethejournalsizeafterthejournalordatafilehasbeencreated.Changingthisparameterwillonlyeffectnewlycreatedjournals.Alsonotethatyoucannotlowerthejournalsizetolessthensizeofthelargestdocumentalreadystoredinthecollection.Note:someothercollectionproperties,suchastype,isVolatile,orkeyOptionscannotbe

Durability

changedoncethecollectioniscreated.

Examples

Readallproperties

arangosh>db.example.properties();

showexecutionresultsChangeaproperty

arangosh>db.example.properties({waitForSync:true});

Per-operationconfiguration

Manydata-modificationoperationsandalsoArangoDB'stransactionsallowtospecifyawaitForSyncattribute,whichwhensetensurestheoperationdatahasbeensynchronizedtodiskwhentheoperationreturns.

Disk-UsageConfiguration

TheamountofdiskspaceusedbyArangoDBisdeterminedbyafewconfigurationoptions.

GlobalConfiguration

ThetotalamountofdiskstoragerequiredbyArangoDBisdeterminedbythesizeofthewrite-aheadlogfilesplusthesizesofthecollectionjournalsanddatafiles.

Therearethefollowingoptionsforconfiguringthenumberandsizesofthewrite-aheadlogfiles:

maximumnumberofreservelogfiles--wal.reserve-logfilesThemaximumnumberofreservelogfilesthatArangoDBwillcreateinabackgroundprocess.Reservelogfilesareusefulinthesituationwhenanoperationneedstobewrittentoalogfilebutthereservespaceinthelogfileistoolowforstoringtheoperation.Inthiscase,anewlogfileneedstobecreatedtostoretheoperation.Creatingnewlogfilesisnormallyslow,soArangoDBwilltrytopre-createlogfilesinabackgroundprocesssotherearealwaysreservelogfileswhentheactivelogfilegetsfull.ThenumberofreservelogfilesthatArangoDBkeepsinthebackgroundisconfigurablewiththisoption.

maximumnumberofhistoriclogfiles--wal.historic-logfilesThemaximumnumberofhistoriclogfilesthatArangoDBwillkeepaftertheyhavebeengarbage-collected.Ifnoreplicationisused,thereisnoneedtokeephistoriclogfilesexceptforhavingalocalchangelog.Inareplicationsetup,thenumberofhistoriclogfilesaffectstheamountofdataaslavecanfetchfromthemaster'slogs.Themorehistoriclogfiles,themorehistoricdataisavailableforaslave,whichisusefuliftheconnectionbetweenmasterandslaveisunstableorslow.Nothavingenoughhistoriclogfilesavailablemightleadtologfiledatabeingdeletedonthemasteralreadybeforeaslavehasfetchedit.

thesizeofeachWALlogfile--wal.logfile-sizeSpecifiesthefilesize(inbytes)foreachwrite-aheadlogfile.Thelogfilesizeshouldbechosensothateachlogfilecanstoreaconsiderableamountofdocuments.Thebiggerthelogfilesizeischosen,thelongeritwilltaketofillupasinglelogfile,whichalsoinfluencesthedelayuntilthedatainalogfilewillbegarbage-collectedandwrittentocollectionjournalsanddatafiles.Italsoaffectshowlonglogfilerecoverywilltakeatserverstart.

whetherornotoversizeentriesareallowed--wal.allow-oversize-entriesWhetherornotitisallowedtostoreindividualdocumentsthatarebiggerthanwouldfitintoasinglelogfile.Settingtheoptiontofalsewillmakesuchoperationsfailwithanerror.Settingtheoptiontotruewillmakesuchoperationssucceed,butwithahighpotentialperformanceimpact.Thereasonisthatforeachoversizeoperation,anindividualoversizelogfileneedstobecreatedwhichmayalsoblockotheroperations.Theoptionshouldbesettofalseifitiscertainthatdocumentswillalwayshaveasizesmallerthanasinglelogfile.Whendatagetscopiedfromthewrite-aheadlogfilesintothejournalsordatafilesofcollections,fileswillbecreatedonthecollectionlevel.Howbigthesefilesareisdeterminedbythefollowingglobalconfigurationvalue:

Durability

--database.maximal-journal-sizesizeMaximalsizeofjournalinbytes.Canbeoverwrittenwhencreatinganewcollection.Notethatthisalsolimitsthemaximalsizeofasingledocument.Thedefaultis32MB.

Per-collectionconfiguration

Thejournalsizecanalsobeadjustedonaper-collectionlevelusingthecollection'spropertiesmethod.

Durability

EncryptionThisfeatureisonlyavailableintheEnterpriseEdition.

WhenyoustoresensitivedatainyourArangoDBdatabase,youwanttoprotectthatdataunderallcircumstances.AtruntimeyouwillprotectitwithSSLtransportencryptionandstrongauthentication,butwhenthedataisalreadyondisk,youalsoneedprotection.ThatiswheretheEncryptionfeaturecomesin.

TheEncryptionfeatureofArangoDBwillencryptalldatathatArangoDBisstoringinyourdatabasebeforeitiswrittentodisk.

ThedataisencryptedwithAES-256-CTR,whichisastrongencryptionalgorithm,thatisverysuitableformulti-processorenvironments.Thismeansthatyourdataissafe,butyourdatabaseisstillfast,evenunderload.

MostmodernCPU'shavebuiltinsupportforhardwareAESencryption,whichmakesitevenfaster.

Note:TheEncryptionfeaturerequirestheRocksDBstorageengine.

Encryptionkeys

TheEncryptionfeatureofArangoDBrequiresasingle32-bytekeyperserver.Itisrecommendedtouseadifferentkeyforeachserver(whenoperatinginaclusterconfiguration).Makesuretoprotectthesekeys!

Thatmeans:

Donotwritethemtopersistentdisksoryourserver(s),alwaysstorethemonanin-memory(tmpfs)filesystem.Transportyourkeyssafelytoyourserver(s).Therearevarioustoolsformanagingsecretslikethis(e.g.vaultproject.io).Storeacopyofyourkeyofflineinasafeplace.Ifyouloseyourkey,thereisNOwaytogetyourdataback.

ConfigurationToactivateencryptionofyourdatabase,youneedtosupplyanencryptionkeytotheserver.

Makesuretopassthisoptiontheveryfirsttimeyoustartyourdatabase.Youcannotencryptadatabasethatalreadyexists.

Note:Youalsohavetoactivatetherocksdbstorageengine.

Encryptionkeystoredinfile

Passthefollowingoptiontoarangod:

$arangod\

--rocksdb.encryption-keyfile=/mytmpfs/mySecretKey\

--server.storage-engine=rocksdb

Thefile/mytmpfs/mySecretKeymustcontaintheencryptionkey.Thisfilemustbesecured,sothatonlyarangodcanaccessit.Youshouldalsoensurethatincasesome-onestealsthehardware,hewillnotbeabletoreadthefile.Forexample,byencryption/mytmpfsorcreatingain-memoryfile-systemunder/mytmpfs.

Encryptionkeygeneratedbyaprogram

Passthefollowingoptiontoarangod:

$arangod\

--rocksdb.encryption-key-generator=path-to-my-generator\

--server.storage-engine=rocksdb

Theprogrampath-to-my-generatoroutputtheencryptiononstandardoutputandexit.

Encryption

Creatingkeys

Theencryptionkeyfilemustcontain32bytesofrandomdata.

Youcancreateitwithacommandlinethis.

ddif=/dev/randombs=1count=32of=yourSecretKeyFile

Forsecurity,itisbesttocreatethesekeysoffline(awayfromyourdatabaseservers)anddirectlystoretheminyousecretmanagementtool.

Encryption

AuditingThisfeatureisavailableintheEnterpriseEdition.

Auditingallowsyoutomonitoraccesstothedatabaseindetail.Ingeneralauditlogsareoftheform

Thetime-stampisinGMT.Thisallowstoeasilymatchlogentriesfromserversindifferenttimezones.

Thenameoftheserver.Youcanspecifyacustomnameonstartup.Otherwisethedefaulthostnameisused.

Theusernameisthe(authenticatedorunauthenticated)namesuppliedbytheclient.Adash-isprintedifnonamewasgivenbytheclient.

Thedatabasedescribesthedatabasethatwasaccessed.Pleasenotethattherearenodatabasecrossingqueries.Eachaccessisrestrictedtoonedatabase.

Theclient-ipdescribesthesourceoftherequest.

Theauthenticationdetailsthemethodsusedtoauthenticatetheuser.

Detailsabouttherequestsfollowintheadditionalfields.

Auditing

AuditConfigurationThisfeatureisavailableintheEnterpriseEdition.

Output--audit.outputoutput

Specifiesthetargetoftheauditlog.Possiblevaluesare

file://filenamewherefilenamecanberelativeorabsolute.

syslog://facilityorsyslog://facility/application-nametologintoasyslogserver.

Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputformultipletargets.

Hostname--audit.hostnamename

Thenameoftheserverusedinauditlogmessages.Bydefaultthesystemhostnameisused.

Configuration

AuditEventsThisfeatureisavailableintheEnterpriseEdition.

Authentication

Unknownauthenticationmethods

2016-10-0315:44:23|server1|-|database1|127.0.0.1:61525|-|unknownauthenticationmethod|/_api/version

Missingcredentials

2016-10-0315:39:49|server1|-|database1|127.0.0.1:61498|-|credentialsmissing|/_api/version

Wrongcredentials

Passwordchangerequired

JWTloginsucceeded

Pleasenote,thattheusergivenasthirdpartistheuserthatrequestedthelogin.Ingeneral,itwillbeempty.

JWTloginfailed

Pleasenote,thattheusergivenasthirdpartistheuserthatrequestedthelogin.Ingeneral,itwillbeempty.

Authorization

Usernotauthorizedtoaccessdatabase

Databases

Createadatabase

Events

Dropadatabase

Collections

Createacollection

2016-10-0517:35:57|server1|user1|database1|127.0.0.1:51294|httpbasic|createcollection'collection1'|ok|/_api

/collection

Truncateacollection

2016-10-0517:36:08|server1|user1|database1|127.0.0.1:51294|httpbasic|truncatecollection'collection1'|ok|/_a

pi/collection/collection1/truncate

Dropacollection

2016-10-0517:36:30|server1|user1|database1|127.0.0.1:51294|httpbasic|deletecollection'collection1'|ok|/_api

/collection/collection1

Indexes

Createaindex

2016-10-0518:19:40|server1|user1|database1|127.0.0.1:52467|httpbasic|createindexin'collection1'|ok|{"field

s":["a"],"sparse":false,"type":"skiplist","unique":false}|/_api/index?collection=collection1

Dropaindex

ction1/44051

Documents

Readingasingledocument

Replacingasingledocument

ion1/21456?ignoreRevs=false

Modifyingasingledocument

on1/21456?keepNull=true&ignoreRevs=false

Events

Deletingasingledocument

on1/21456?ignoreRevs=false

Forexample,ifsomeonestriestodeleteanon-existingdocument,itwillbeloggedas

ection1/21456?ignoreRevs=false

Queries

eturni|/_api/cursor

Events

IntroductiontoReplicationReplicationallowsyoutoreplicatedataontoanothermachine.ItformsthebaseofalldisasterrecoveryandfailoverfeaturesArangoDBoffers.

ArangoDBoffersasynchronousandsynchronousreplicationwhichbothhavetheirprosandcons.Bothmodesmayandshouldbecombinedinarealworldscenarioandbeappliedintheusecasewheretheyexcelmost.

Wewilldescribeprosandconsofeachoftheminthefollowingsections.

Synchronousreplication

Synchronousreplicationonlyworkswithinaclusterandistypicallyusedformissioncriticaldatawhichmustbeaccessibleatalltimes.Synchronousreplicationgenerallystoresacopyofashard'sdataonanotherdbserverandkeepsitinsync.Essentially,whenstoringdataafterenablingsynchronousreplicationtheclusterwillwaitforallreplicastowriteallthedatabeforegreenlightingthewriteoperationtotheclient.Thiswillnaturallyincreasethelatencyabit,sinceonemorenetworkhopisneededforeachwrite.However,itwillenabletheclustertoimmediatelyfailovertoareplicawheneveranoutagehasbeendetected,withoutlosinganycommitteddata,andmostlywithoutevensignalinganerrorconditiontotheclient.

Synchronousreplicationisorganizedsuchthateveryshardhasaleaderandr-1followers,whererdenotedthereplicationfactor.ThenumberoffollowerscanbecontrolledusingthereplicationFactorparameterwheneveryoucreateacollection,thereplicationFactorparameteristhetotalnumberofcopiesbeingkept,thatis,itisoneplusthenumberoffollowers.

Satellitecollections

SatellitecollectionsaresynchronouslyreplicatedcollectionshavingadynamicreplicationFactor.Theywillreplicatealldatatoalldatabaseserversallowingthedatabaseserverstojoindatalocallyinsteadofdoingheavynetworkoperations.

Satellitecollectionsareanenterpriseonlyfeature.

Asynchronousreplication

InArangoDBanywriteoperationwillbeloggedtothewrite-aheadlog.WhenusingAsynchronousreplicationslaveswillconnecttoamasterandapplyalltheeventsfromtheloginthesameorderlocally.Afterthat,theywillhavethesamestateofdataasthemasterdatabase.

Replication

AsynchronousreplicationAsynchronousreplicationworksbyloggingeverydatamodificationonamasterandreplayingtheseeventsonanumberofslaves.

Transactionsarehonoredinreplication,i.e.transactionalwriteoperationswillbecomevisibleonslavesatomically.

Asallwriteoperationswillbeloggedtoamasterdatabase'swrite-aheadlog,thereplicationinArangoDBcurrentlycannotbeusedforwrite-scaling.ThemainpurposesofthereplicationincurrentArangoDBaretoprovideread-scalabilityand"hotbackups"ofspecificdatabases.

Itispossibletoconnectmultipleslavedatabasestothesamemasterdatabase.Slavedatabasesshouldbeusedasread-onlyinstances,andnouser-initiatedwriteoperationsshouldbecarriedoutonthem.Otherwisedataconflictsmayoccurthatcannotbesolvedautomatically,andthatwillmakethereplicationstop.

Inanasynchronousreplicationscenarioslaveswillpullchangesfromthemasterdatabase.Slavesneedtoknowtowhichmasterdatabasetheyshouldconnectto,butamasterdatabaseisnotawareoftheslavesthatreplicatefromit.Whenthenetworkconnectionbetweenthemasterdatabaseandaslavegoesdown,writeoperationsonthemastercancontinuenormally.Whenthenetworkisupagain,slavescanreconnecttothemasterdatabaseandtransfertheremainingchanges.Thiswillhappenautomaticallyprovidedslavesareconfiguredappropriately.

Replicationlag

Inthissetup,writeoperationsareappliedfirstinthemasterdatabase,andappliedintheslavedatabase(s)afterwards.

Forexample,let'sassumeawriteoperationisexecutedinthemasterdatabaseatpointintimet0.Tomakeaslavedatabaseapplythesameoperation,itmustfirstfetchthewriteoperation'sdatafrommasterdatabase'swrite-aheadlog,thenparseitandapplyitlocally.Thiswillhappenatsomepointintimeaftert0,let'ssayt1.

Thedifferencebetweent1andt0iscalledthereplicationlag,anditisunavoidableinasynchronousreplication.Theamountofreplicationlagdependsonmanyfactors,afewofwhichare:

thenetworkcapacitybetweentheslavesandthemastertheloadofthemasterandtheslavesthefrequencyinwhichslavespollthemasterforupdates

Betweent0andt1,thestateofdataonthemasterisnewerthanthestateofdataontheslave(s).Atpointintimet1,thestateofdataonthemasterandslave(s)isconsistentagain(providednonewdatamodificationshappenedonthemasterinbetween).Thus,thereplicationwillleadtoaneventuallyconsistentstateofdata.

Replicationconfiguration

Thereplicationisturnedoffbydefault.Inordertocreateamaster-slavesetup,theso-calledreplicationapplierneedstobeenabledontheslavedatabases.

Replicationisconfiguredonaper-databaselevel.Ifmultipledatabasearetobereplicated,thereplicationmustbesetupindividuallyperdatabase.

Thereplicationapplierontheslavecanbeusedtoperformaone-timesynchronizationwiththemaster(andthenstop),ortoperformanongoingreplicationofchanges.Toresumereplicationonslaverestart,theautoStartattributeofthereplicationappliermustbesettotrue.

Replicationoverhead

Asthemasterserversarelogginganywriteoperationinthewrite-ahead-loganywayreplicationdoesn'tcauseanyextraoverheadonthemaster.Howeveritwillofcoursecausesomeoverheadforthemastertoserveincomingreadrequestsoftheslaves.Returningtherequesteddataishoweveratrivialtaskforthemasterandshouldnotresultinanotableperformancedegrationinproduction.

Components

ReplicationLogger

Purpose

Thereplicationloggerwillwritealldata-modificationoperationsintothewrite-aheadlog.Thislogmaythenbereadbyclientstoreplayanydatamodificationonadifferentserver.

Checkingthestate

Toquerythecurrentstateofthelogger,usethestatecommand:

require("@arangodb/replication").logger.state();

Theresultmightlooklikethis:

"state":{

"running":true,

"lastLogTick":"133322013",

"totalEvents":16,

"time":"2014-07-06T12:58:11Z"

"server":{

"version":"2.2.0-devel",

"serverId":"40897075811372"

"clients":{

Therunningattributewillalwaysbetrue.InearlierversionsofArangoDBthereplicationwasoptionalandthiscouldhavebeenfalse.

ThetotalEventsattributeindicateshowmanylogeventshavebeenloggedsincethestartoftheArangoDBserver.Finally,thelastLogTickvalueindicatestheidofthelastoperationthatwaswrittentotheserver'swrite-aheadlog.Itcanbeusedtodeterminewhethernewoperationswerelogged,andisalsousedbythereplicationapplierforincrementalfetchingofdata.

Note:ThereplicationloggerstatecanalsobequeriedviatheHTTPAPI.

Toquerywhichdatarangesarestillavailableforreplicationclientstofetch,theloggerprovidesthefirstTickandtickRangesfunctions:

require("@arangodb/replication").logger.firstTick();

ThiswillreturntheminimumtickvaluethattheservercanprovidetoreplicationclientsviaitsreplicationAPIs.ThetickRangesfunctionreturnstheminimumandmaximumtickvaluesperlogfile:

require("@arangodb/replication").logger.tickRanges();

ReplicationApplier

Purpose

Thepurposeofthereplicationapplieristoreaddatafromamasterdatabase'seventlog,andapplythemlocally.Theapplierwillcheckthemasterdatabasefornewoperationsperiodically.Itwillperformanincrementalsynchronization,i.e.onlyaskingthemasterforoperationsthatoccurredafterthelastsynchronization.

Thereplicationapplierdoesnotgetnotifiedbythemasterdatabasewhenthereare"new"operationsavailable,butinsteadusesthepullprinciple.Itmightthustakesometime(theso-calledreplicationlag)beforeanoperationfromthemasterdatabasegetsshippedtoandappliedinaslavedatabase.

Thereplicationapplierofadatabaseisruninaseparatethread.Itmayencounterproblemswhenanoperationfromthemastercannotbeappliedsafely,orwhentheconnectiontothemasterdatabasegoesdown(networkoutage,masterdatabaseisdownorunavailableetc.).Inthiscase,thedatabase'sreplicationapplierthreadmightterminateitself.Itisthenuptotheadministratortofixtheproblemandrestartthedatabase'sreplicationapplier.

Ifthereplicationappliercannotconnecttothemasterdatabase,orthecommunicationfailsatsomepointduringthesynchronization,thereplicationapplierwilltrytoreconnecttothemasterdatabase.Itwillgiveupreconnectingonlyafteraconfigurableamountofconnectionattempts.

Thereplicationapplierstateisqueryableatanytimebyusingthestatecommandoftheapplier.Thiswillreturnthestateoftheapplierofthecurrentdatabase:

require("@arangodb/replication").applier.state();

"state":{

"running":true,

"lastAppliedContinuousTick":"152786205",

"lastProcessedContinuousTick":"152786205",

"lastAvailableContinuousTick":"152786205",

"progress":{

"time":"2014-07-06T13:04:57Z",

"message":"fetchingmasterlogfromoffset152786205",

"failedConnects":0

"totalRequests":38,

"totalFailedConnects":0,

"totalEvents":1,

"lastError":{

"errorNum":0

"time":"2014-07-06T13:04:57Z"

"server":{

"version":"2.2.0-devel",

"serverId":"210189384542896"

"endpoint":"tcp://master.example.org:8529",

"database":"_system"

Therunningattributeindicateswhetherthereplicationapplierofthecurrentdatabaseiscurrentlyrunningandpollingtheserveratendpointfornewevents.

Theprogress.failedConnectsattributeshowshowmanyfailedconnectionattemptsthereplicationappliercurrentlyhasencounteredinarow.Incontrast,thetotalFailedConnectsattributeindicateshowmanyfailedconnectionattemptstheapplierhasmadeintotal.ThetotalRequestsattributeshowshowmanyrequeststheapplierhassenttothemasterdatabaseintotal.ThetotalEventsattributeshowshowmanylogeventstheapplierhasreadfromthemaster.

Theprogress.messagesub-attributeprovidesabriefhintofwhattheappliercurrentlydoes(ifitisrunning).ThelastErrorattributealsohasanoptionalerrorMessagesub-attribute,showingthelatesterrormessage.TheerrorNumsub-attributeofthelastErrorattributecanbeusedbyclientstoprogrammaticallycheckforerrors.Itshouldbe0ifthereisnoerror,anditshouldbenon-zeroiftheapplierterminateditselfduetoaproblem.

Hereisanexampleofthestateafterthereplicationapplierterminateditselfdueto(repeated)connectionproblems:

"state":{

"running":false,

"progress":{

"time":"2014-07-06T13:14:37Z",

"message":"applierstopped",

"failedConnects":6

"totalRequests":79,

"totalFailedConnects":11,

"totalEvents":0,

"lastError":{

"time":"2014-07-06T13:09:41Z",

"errorMessage":"couldnotconnecttomasterattcp://master.example.org:8529:Couldnotconnectto'tcp:/...",

"errorNum":1400

Note:thestateofadatabase'sreplicationapplierisqueryableviatheHTTPAPI,too.PleaserefertoHTTPInterfaceforReplicationformoredetails.

All-in-onesetup

Tocopytheinitialdatafromtheslavetothemasterandstartthecontinuousreplication,thereisanall-in-onecommandsetupReplication:

require("@arangodb/replication").setupReplication(configuration);

Thefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthe_systemdatabase.Notethatitshouldberunontheslaveandnotthemaster:

db._useDatabase("_system");

require("@arangodb/replication").setupReplication({

endpoint:"tcp://master.domain.org:8529",

username:"myuser",

password:"mypasswd",

verbose:false,

includeSystem:false,

incremental:true,

autoResync:true

Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationisstarted,orincasetheinitialsynchronizationhasfailed.

Iftheinitialsynchronizationissuccessful,thecommandwillstorethegivenconfigurationontheslave.Italsoconfiguresthecontinuousreplicationtostartautomaticallyiftheslaveisrestarted,i.e.autoStartissettotrue.

Ifthecommandisrunwhiletheslave'sreplicationapplierisalreadyrunning,itwillfirststoptherunningapplier,dropitsconfigurationanddoaresynchronizationofdatawiththemaster.Itwillthenusetheprovidedconfigration,overwritinganypreviouslyexistingreplicationconfigurationontheslave.

StartingandStopping

Tomanuallystartandstoptheapplierinthecurrentdatabase,thestartandstopcommandscanbeusedlikethis:

require("@arangodb/replication").applier.start(<tick>);

require("@arangodb/replication").applier.stop();

Note:Startingareplicationapplierwithoutsettingupaninitialconfigurationwillfail.ThereplicationapplierwilllookforitsconfigurationinafilenamedREPLICATION-APPLIER-CONFIGinthecurrentdatabase'sdirectory.Ifthefileisnotpresent,ArangoDBwillusesomedefaultconfiguration,butitcannotguesstheendpoint(theaddressofthemasterdatabase)theappliershouldconnectto.Thusstartingtheapplierwithoutconfigurationwillfail.

Notethatatthefirsttimeyoustarttheapplier,youshouldpassthevaluereturnedinthelastLogTickattributeoftheinitialsyncoperation.

Note:Startingadatabase'sreplicationapplierviathestartcommandwillnotnecessarilystarttheapplieronthenextandfollowingArangoDBserverrestarts.Additionally,stoppingadatabase'sreplicationappliermanuallywillnotnecessarilypreventtheapplierfrombeingstartedagainonthenextserverstart.Allofthisisconfigurableseparately(hangonreading).

Note:whenstoppingandrestartingthereplicationapplierofdatabase,itwillresumewhereitlaststopped.Thisissensiblebecausereplicationlogeventsshouldbeappliedincrementally.Ifthereplicationapplierofadatabasehasneverbeenstartedbefore,itneedssometickvaluefromthemaster'slogfromwhichtostartfetchingevents.

Thereisonecaveattoconsiderwhenstoppingareplicationontheslave:iftherearestillongoingreplicatedtransactionsthatareneithercommittedoraborted,stoppingthereplicationapplierwillcausetheseoperationstobelostfortheslave.Ifthesetransactionscommitonthemasterlaterandthereplicationisresumed,theslavewillnotbeabletocommitthesetransactions,too.Thusstoppingthereplicationapplierontheslavemanuallyshouldonlybedoneifthereiscertaintythattherearenoongoingtransactionsonthemaster.

Configuration

Toconfigurethereplicationapplierofaspecificdatabase,usethepropertiescommand.Usingitwithoutanyargumentswillreturntheapplier'scurrentconfiguration:

require("@arangodb/replication").applier.properties();

"requestTimeout":600,

"connectTimeout":10,

"ignoreErrors":0,

"maxConnectRetries":10,

"chunkSize":0,

"autoStart":false,

"adaptivePolling":true,

"includeSystem":true,

"requireFromPresent":false,

"autoResync":false,

"autoResyncRetries":2,

"verbose":false

Note:Thereisnoendpointattributeconfiguredyet.Theendpointattributeisrequiredforthereplicationappliertobestartable.Youmayalsowanttoconfigureausernameandpasswordfortheconnectionviatheusernameandpasswordattributes.

require("@arangodb/replication").applier.properties({

username:"root",

password:"secret",

verbose:false

Thiswillre-configurethereplicationapplierforthecurrentdatabase.Theconfigurationwillbeusedfromthenextstartofthereplicationapplier.Thereplicationappliercannotbere-configuredwhileitisrunning.Itmustbestoppedfirsttobere-configured.

TomakethereplicationapplierofthecurrentdatabasestartautomaticallywhentheArangoDBserverstarts,usetheautoStartattribute.

SettingtheadaptivePollingattributetotruewillmakethereplicationapplierpollthemasterdatabaseforchangeswithavariablefrequency.Thereplicationapplierwillthenlowerthefrequencywhenthemasterisidle,andincreaseitwhenthemastercanprovidenewevents).Otherwisethereplicationapplierwillpollthemasterdatabaseforchangeswithaconstantfrequency.

TheidleMinWaitTimeattributecontrolstheminimumwaittime(inseconds)thatthereplicationapplierwillintentionallyidlebeforefetchingmorelogdatafromthemasterincasethemasterhasalreadysentallitslogdata.ThiswaittimecanbeusedtocontrolthefrequencywithwhichthereplicationappliersendsHTTPlogfetchrequeststothemasterincasethereisnowriteactivityonthemaster.

TheidleMaxWaitTimeattributecontrolsthemaximumwaittime(inseconds)thatthereplicationapplierwillintentionallyidlebeforefetchingmorelogdatafromthemasterincasethemasterhasalreadysentallitslogdataandtherehavebeenpreviouslogfetchattemptsthatresultedinnomorelogdata.Thiswaittimecanbeusedtocontrolthemaximumfrequencywithwhichthereplicationappliersends

HTTPlogfetchrequeststothemasterincasethereisnowriteactivityonthemasterforlongerperiods.NotethatthisconfigurationvaluewillonlybeusediftheoptionadaptivePollingissettotrue.

Tosetatimeoutforconnectionandfollowingrequestattempts,usetheconnectTimeoutandrequestTimeoutvalues.ThemaxConnectRetriesattributeconfiguresafterhowmanyfailedconnectionattemptsinarowthereplicationapplierwillgiveupandturnitselfoff.Youmaywanttosetthistoahighvaluesothattemporarynetworkoutagesdonotleadtothereplicationapplierstoppingitself.TheconnectRetryWaitTimeattributeconfigureshowlongthereplicationapplierwillwaitbeforeretryingtheconnectiontothemasterincaseofconnectionproblems.

ThechunkSizeattributecanbeusedtocontroltheapproximatemaximumsizeofamaster'sresponse(inbytes).Settingittoalowvaluemaymakethemasterrespondfaster(lessdataisassembledbeforethemastersendstheresponse),butmayrequiremorerequest-responseroundtrips.Setitto0touseArangoDB'sbuilt-indefaultvalue.

TheincludeSystemattributecontrolswhetherchangestosystemcollections(suchas_graphsor_users)shouldbeapplied.Ifsettotrue,changesinthesecollectionswillbereplicated,otherwise,theywillnotbereplicated.Itisoftennotnecessarytoreplicatedatafromsystemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausetheslaveneedstohaveitsownsystemcollectionsinordertostartandkeepoperational.

TherequireFromPresentattributecontrolswhethertheapplierwillstartsynchronizingincaseitdetectsthatthemastercannotprovidedatafortheinitialtickvalueprovidedbytheslave.ThismaybethecaseifthemasterdoesnothaveabigenoughbacklogofhistoricWALlogfiles,andwhenthereplicationisre-startedafteralongerpause.WhenrequireFromPresentissettotrue,thenthereplicationapplierwillcheckatstartwhetherthestarttickfromwhichitstartsorresumesreplicationisstillpresentonthemaster.Ifnot,thentherewouldbedataloss.IfrequireFromPresentistrue,thereplicationapplierwillabortwithanappropriateerrormessage.Ifsettofalse,thenthereplicationapplierwillstillstart,andignorethedataloss.

TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentandautoResyncaresettotrueandthemastercannotprovidethelogdatatheslaverequests,thereplicationapplierwillstopasusual.ButduetothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,thereplicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullre-synchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.

Automaticre-synchronizationmaytransferalotofdatafromthemastertotheslaveandcanbeexpensive.Itisthereforeturnedoffbydefault.Whenturnedoff,theslavewillneverperformanautomaticre-synchronizationwiththemaster.

TheautoResyncRetriesoptioncanbeusedtocontrolthenumberofresynchronizationretriesthatwillbeperformedinarowwhenautomaticresynchronizationisenabledandkicksin.Settingthisto0willeffectivelydisableautoResync.Settingittosomeothervaluewilllimitthenumberofretriesthatareperformed.Thishelpspreventingendlessretriesincaseresynchronizationsalwaysfail.

Theverboseattributecontrolstheverbosityofthereplicationlogger.Settingittotruewillmakethereplicationapplierwritealinetothelogforeveryoperationitperforms.Thisshouldonlybeusedfordiagnosingreplicationproblems.

Thefollowingexamplewillsetmostofthediscussedpropertiesforthecurrentdatabase'sapplier:

username:"root",

password:"secret",

adaptivePolling:true,

connectTimeout:15,

maxConnectRetries:100,

chunkSize:262144,

autoStart:true,

includeSystem:true,

autoResync:true,

autoResyncRetries:2,

Aftertheapplierisnowfullyconfigured,itcouldtheoreticallybestarted.However,wemayfirstneedaninitialsynchronizationofallcollectionsandtheirdatafromthemasterbeforewestartthereplicationapplier.

Theonlysafemethodfordoingafullsynchronization(orre-synchronization)isthusto

stopthereplicationapplierontheslave(ifcurrentlyrunning)performaninitialfullsyncwiththemasterdatabasenotethemasterdatabase'slastLogTickvalueand

startthecontinuousreplicationapplierontheslaveusingthistickvalue.

Theinitialsynchronizationforthecurrentdatabaseisexecutedwiththesynccommand:

require("@arangodb/replication").sync({

username:"root",

password:"secret,

includeSystem:true

TheincludeSystemoptioncontrolswhetherdatafromsystemcollections(suchas_graphsand_users)shallbesynchronized.

TheinitialsynchronizationcanoptionallybeconfiguredtoincludeorexcludespecificcollectionsusingtherestrictTypeandrestrictCollectionparameters.

Thefollowingcommandonlysynchronizescollectionfooandbar:

username:"root",

password:"secret,

restrictType:"include",

restrictCollections:["foo","bar"]

UsingarestrictTypeofexclude,allcollectionsbutthespecifiedwillbesynchronized.

Warning:syncwilldoafullsynchronizationofthecollectionsinthecurrentdatabasewithcollectionspresentinthemasterdatabase.Anylocalinstancesofthecollectionsandalltheirdataareremoved!Onlyexecutethiscommandifyouaresureyouwanttoremovethelocaldata!

Assyncdoesafullsynchronization,itmighttakeawhiletoexecute.Whensynccompletessuccessfully,itreturnsanarrayofcollectionsithassynchronizedinitscollectionsattribute.Itwillalsoreturnthemasterdatabase'slastlogtickvalueatthetimethesyncwasstartedonthemaster.ThetickvalueiscontainedinthelastLogTickattributeofthesynccommand:

"lastLogTick":"231848833079705",

"collections":[...]

Nowyoucanstartthecontinuoussynchronizationforthecurrentdatabaseontheslavewiththecommand

require("@arangodb/replication").applier.start("231848833079705");

Note:Thetickvaluesshouldbetreatedasstrings.Usingnumericdatatypesfortickvaluesisunsafebecausetheymightexceedthe32bitvalueandtheIEEE754doubleaccuracyranges.

Per-DatabaseSetupThispagedescribesthereplicationprocessbasedonaspecificdatabasewithinanArangoDBinstance.Thatmeansthatonlythespecifieddatabasewillbereplicated.

Settingupaworkingmaster-slavereplicationrequirestwoArangoDBinstances:

master:thisistheinstancethatalldata-modificationoperationsshouldbedirectedtoslave:onthisinstance,we'llstartareplicationapplier,andthiswillfetchdatafromthemasterdatabase'swrite-aheadlogandapplyitsoperationslocally

Forthefollowingexamplesetup,we'llusetheinstancetcp://master.domain.org:8529asthemaster,andtheinstancetcp://slave.domain.org:8530asaslave.

Thegoalistohavealldatafromthedatabase_systemonmastertcp://master.domain.org:8529bereplicatedtothedatabase_systemontheslavetcp://slave.domain.org:8530.

Onthemaster,nothingspecialneedstobedone,asallwriteoperationswillautomaticallybeloggedinthemaster'swrite-aheadlog(WAL).

All-in-onesetup

Tomakethereplicationcopytheinitialdatafromthemastertotheslaveandstartthecontinuousreplicationontheslave,thereisanall-in-onecommand:

require("@arangodb/replication").setupReplication(configuration);

Thefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthe_systemdatabase.Notethatitshouldberunontheslaveandnotthemaster:

require("@arangodb/replication").setupReplication({

username:"myuser",

verbose:false,

incremental:true,

autoResync:true

Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationhasbeenstarted,orincasetheinitialsynchronizationhasfailed.

Initialsynchronization

Theinitialsynchronizationandcontinuousreplicationappliercanalsobestartedseparately.Tostartreplicationontheslave,makesuretherecurrentlyisnoreplicationapplierrunning.

Thefollowingcommandsstoparunningapplierintheslave's_systemdatabase:

require("@arangodb/replication").applier.stop();

Thestopoperationwillterminateanyreplicationactivityinthe_systemdatabaseontheslave.

Afterthat,theinitialsynchronizationcanberun.Itwillcopythecollectionsfromthemastertotheslave,overwritingexistingdata.Toruntheinitialsynchronization,executethefollowingcommandsontheslave:

username:"myuser",

verbose:false

Usernameandpasswordonlyneedtobespecifiedwhenthemasterrequiresauthentication.Tocheckwhatthesynchronizationiscurrentlydoing,supplysettheverboseoptiontotrue.Ifset,thesynchronizationwillcreatelogmessageswiththecurrentsynchronizationstatus.

Warning:Thesynccommandwillreplacedataintheslavedatabasewithdatafromthemasterdatabase!Onlyexecutethesecommandsifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!

ThesyncoperationwillreturnanattributenamedlastLogTickwhichwe'llneedtonote.Thelastlogtickwillbeusedasthestartingpointforsubsequentreplicationactivity.Let'sassumewegotthefollowinglastlogtick:

InitialsynchronizationfromtheArangoShell

Theinitialsynchronizationviathesynccommandmaytakealongtimetocomplete.Theshellwillblockuntiltheslavehascompletedtheinitialsynchronizationoruntilanerroroccurs.Bydefault,thesynccommandintheArangoShellwillpolltheslaveforastatusupdateevery10seconds.

Optionallythesynccommandcanbemadenon-blockingbysettingitsasyncoptiontotrue.Inthiscase,thesynccommandwillreturninstantlywithanidstring,andtheinitialsynchronizationwillrundetachedonthemaster.TofetchthecurrentstatusofthesyncprogressfromtheArangoShell,thegetSyncResultfunctioncanbeusedasfollows:

varreplication=require("@arangodb/replication");

/*runcommandinasyncmode*/

varid=replication.sync({

username:"myuser",

async:true

/*nowquerythestatusofouroperation*/

print(replication.getSyncResult(id));

getSyncResultwillreturnfalseaslongasthesynchronizationisnotcomplete,andreturnthesynchronizationresultotherwise.

Continuoussynchronization

Whentheinitialsynchronizationisfinished,thecontinuousreplicationappliercanbestartedusingthelastlogtickprovidedbythesynccommand.Beforestartingit,thereisatleastoneconfigurationoptiontoconsider:replicationontheslavewillberunninguntiltheslavegetsshutdown.Whentheslaveservergetsrestarted,replicationwillbeturnedoffagain.Tochangethis,wefirstneedtoconfiguretheslave'sreplicationapplierandsetitsautoStartattribute.

Here'sthecommandtoconfigurethereplicationapplierwithseveraloptions,includingtheautoStartattribute:

username:"myuser",

autoStart:true,

autoResync:true,

requireFromPresent:false,

idleMinWaitTime:0.5,

idleMaxWaitTime:1.5,

verbose:false

Animportantconsiderationforreplicationiswhetherdatafromsystemcollections(suchas_graphsor_users)shouldbeapplied.TheincludeSystemoptioncontrolsthat.Ifsettotrue,changesinsystemcollectionswillbereplicated.Otherwise,theywillnotbereplicated.Itisoftennotnecessarytoreplicatedatafromsystemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausetheslaveneedstohaveitsownsystemcollectionsinordertostartandkeepoperational.

TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentandautoResyncaresettotrueandthemastercannotprovidethelogdatatheslavehadrequested,thereplicationapplierwillstopasusual.ButduetothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,thereplicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullre-synchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.

Notethatautomaticre-synchronization(autoResyncoptionsettotrue)maytransferalotofdatafromthemastertotheslaveandcanthereforebeexpensive.Stillit'sturnedonheresothere'slessneedformanualintervention.

Nowit'stimetostartthereplicationapplierontheslaveusingthelastlogtickwegotbefore:

require("@arangodb/replication").applier.start("40694126");

Thiswillreplicatealloperationshappeninginthemaster'ssystemdatabaseandapplythemontheslave,too.

Afterthat,youshouldbeabletomonitorthestateandprogressofthereplicationapplierbyexecutingthestatecommandontheslaveserver:

require("@arangodb/replication").applier.state();

Pleasenotethatstoppingthereplicationapplierontheslaveusingthestopcommandshouldbeavoided.Thereasonisthatcurrentlyongoingtransactions(thathavepartlybeenreplicatedtotheslave)willbeneedtoberestartedafterarestartofthereplicationapplier.Stoppingandrestartingthereplicationapplierontheslaveshouldthusonlybeperformedifthereiscertaintythatthemasteriscurrentlyfullyidleandalltransactionshavebeenreplicatedfully.

Notethatwhileaslavehasonlypartlyexecutedatransactionfromthemaster,itmightkeepawritelockonthecollectionsinvolvedinthetransaction.

YoumayalsowanttocheckthemasterandslavestatesviatheHTTPAPIs(seeHTTPInterfaceforReplication).

Server-levelSetupThispagedescribesthereplicationprocessbasedonacompleteArangoDBinstance.Thatmeansthatallincludeddatabaseswillbereplicated.

Settingupaworkingmaster-slavereplicationrequirestwoArangoDBinstances:

master:thisistheinstancethatalldata-modificationoperationsshouldbedirectedtoslave:onthisinstance,we'llstartareplicationapplier,andthiswillfetchdatafromthemasterdatabase'swrite-aheadlogandapplyitsoperationslocally

Thegoalistohavealldataofalldatabasesonmastertcp://master.domain.org:8529bereplicatedtotheslaveinstancetcp://slave.domain.org:8530.

Onthemaster,nothingspecialneedstobedone,asallwriteoperationswillautomaticallybeloggedinthemaster'swrite-aheadlog(WAL).

All-in-onesetup

Tomakethereplicationcopytheinitialdatafromthemastertotheslaveandstartthecontinuousreplicationontheslave,thereisanall-in-onecommand:

require("@arangodb/replication").setupReplicationGlobal(configuration);

ThefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthecompleteArangoDBinstance.Notethatitshouldberunontheslaveandnotthemaster:

require("@arangodb/replication").setupReplicationGlobal({

endpoint:"tcp://127.0.0.1:8529",

username:"root",

password:"",

autoStart:true

Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationhasbeenstarted,orincasetheinitialsynchronizationhasfailed.

Stoppingsynchronization

Theinitialsynchronizationandcontinuousreplicationappliercanalsobestartedseparately.Tostartreplicationontheslave,makesuretherecurrentlyisnoreplicationapplierrunning.

Thefollowingcommandsstoparunningapplierintheslave'sinstance:

require("@arangodb/replication").globalApplier.stop();

ThestopoperationwillterminateanyreplicationactivityintheArangoDBinstanceontheslave.

Afterthat,theinitialsynchronizationcanberun.Itwillcopythecollectionsfromthemastertotheslave,overwritingexistingdata.Toruntheinitialsynchronization,executethefollowingcommandsontheslave:

require("@arangodb/replication").syncGlobal({

username:"myuser",

verbose:false

Usernameandpasswordonlyneedtobespecifiedwhenthemasterrequiresauthentication.Tocheckwhatthesynchronizationiscurrentlydoing,supplysettheverboseoptiontotrue.Ifset,thesynchronizationwillcreatelogmessageswiththecurrentsynchronizationstatus.

Warning:ThesyncGlobalcommandwillreplacedataintheslavedatabasewithdatafromthemasterdatabase!Onlyexecutethesecommandsifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!

ThesyncoperationwillreturnanattributenamedlastLogTickwhichwe'llneedtonote.Thelastlogtickwillbeusedasthestartingpointforsubsequentreplicationactivity.Let'sassumewegotthefollowinglastlogtick:

InitialsynchronizationfromtheArangoShellTheinitialsynchronizationviathesyncGlobalcommandmaytakealongtimetocomplete.Theshellwillblockuntiltheslavehascompletedtheinitialsynchronizationoruntilanerroroccurs.Bydefault,thesyncGlobalcommandintheArangoShellwillpolltheslaveforastatusupdateevery10seconds.

OptionallythesyncGlobalcommandcanbemadenon-blockingbysettingitsasyncoptiontotrue.Inthiscase,thesyncGlobalcommandwillreturninstantlywithanidstring,andtheinitialsynchronizationwillrundetachedonthemaster.TofetchthecurrentstatusofthesyncGlobalprogressfromtheArangoShell,thegetSyncResultfunctioncanbeusedasfollows:

varid=replication.syncGlobal({

username:"myuser",

async:true

Continuoussynchronization

Whentheinitialsynchronizationisfinished,thecontinuousreplicationappliercanbestartedusingthelastlogtickprovidedbythesyncGlobalcommand.Beforestartingit,thereisatleastoneconfigurationoptiontoconsider:replicationontheslavewillberunninguntiltheslavegetsshutdown.Whentheslaveservergetsrestarted,replicationwillbeturnedoffagain.Tochangethis,wefirstneedtoconfiguretheslave'sreplicationapplierandsetitsautoStartattribute.

Here'sthecommandtoconfigurethereplicationapplierwithseveraloptions,includingtheautoStartattribute:

require("@arangodb/replication").globalApplier.properties({

username:"myuser",

autoStart:true,

autoResync:true,

requireFromPresent:false,

idleMinWaitTime:0.5,

idleMaxWaitTime:1.5,

verbose:false

Animportantconsiderationforreplicationiswhetherdatafromsystemcollections(suchas_graphsor_users)shouldbeapplied.TheincludeSystemoptioncontrolsthat.Ifsettotrue,changesinsystemcollectionswillbereplicated.Otherwise,theywillnotbereplicated.Itisoftennotnecessarytoreplicatedatafromsystemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausetheslaveneedstohaveitsownsystemcollectionsinordertostartandkeepoperational.

TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentandautoResyncaresettotrueandthemastercannotprovidethelogdatatheslavehadrequested,thereplicationapplierwillstopasusual.ButduetothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,thereplicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullre-synchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.

Notethatautomaticre-synchronization(autoResyncoptionsettotrue)maytransferalotofdatafromthemastertotheslaveandcanthereforebeexpensive.Stillit'sturnedonheresothere'slessneedformanualintervention.

Nowit'stimetostartthereplicationapplierontheslaveusingthelastlogtickwegotbefore:

require("@arangodb/replication").globalApplier.start("40694126");

Thiswillreplicatealloperationshappeninginthemaster'ssystemdatabaseandapplythemontheslave,too.

Afterthat,youshouldbeabletomonitorthestateandprogressofthereplicationapplierbyexecutingthestatecommandontheslaveserver:

require("@arangodb/replication").globalApplier.state();

Pleasenotethatstoppingthereplicationapplierontheslaveusingthestopcommandshouldbeavoided.Thereasonisthatcurrentlyongoingtransactions(thathavepartlybeenreplicatedtotheslave)willbeneedtoberestartedafterarestartofthereplicationapplier.Stoppingandrestartingthereplicationapplierontheslaveshouldthusonlybeperformedifthereiscertaintythatthemasteriscurrentlyfullyidleandalltransactionshavebeenreplicatedfully.

Notethatwhileaslavehasonlypartlyexecutedatransactionfromthemaster,itmightkeepawritelockonthecollectionsinvolvedinthetransaction.

YoumayalsowanttocheckthemasterandslavestatesviatheHTTPAPIs(seeHTTPInterfaceforReplication).

SyncingCollectionsInordertosynchronizedataforasinglecollectionfromamastertoaslaveinstance,thereisthesyncCollectionfunction:

Itwillfetchalldocumentsofthespecifiedcollectionfromthemasterdatabaseandstoretheminthelocalinstance.Afterthesynchronization,thecollectiondataontheslavewillbeidenticaltothedataonthemaster,providednofurtherdatachangeshappenonthemaster.AnydatachangesthatareperformedonthemasterafterthesynchronizationwasstartedwillnotbecapturedbysyncCollection,butneedtobereplicatedusingtheregularreplicationappliermechanism.

Thegoalistohavealldatafromthecollectiontestindatabase_systemonmastertcp://master.domain.org:8529bereplicatedtothecollectiontestindatabase_systemontheslavetcp://slave.domain.org:8530.

Onthemaster,thecollectiontestneedstobepresentinthe_systemdatabase,withanydatainit.

Totransferthiscollectiontotheslave,issuethefollowingcommandsthere:

require("@arangodb/replication").syncCollection("test",{

username:"myuser",

password:"mypasswd"

Warning:ThesyncCollectioncommandwillreplacethecollection'sdataintheslavedatabasewithdatafromthemasterdatabase!Onlyexecutethesecommandsifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!

SettingtheoptionalincrementalattributeinthecalltosyncCollectionwillstartanincrementaltransferofdata.Thismaybeusefulincasewhentheslavealreadyhaspartsoralmostallofthedatainthecollectionandonlythedifferencesneedtobesynchronized.Notethattocomputethedifferencestheincrementaltransferwillbuildasortedlistofalldocumentkeysinthecollectiononboththeslaveandthemaster,whichmaystillbeexpensiveforhugecollectionsintermsofmemoryusageandruntime.Duringbuildingthelistofkeysthecollectionwillberead-lockedonthemaster.

TheinitialSyncMaxWaitTimeattributeinthecalltosyncCollectioncontrolshowlongtheslavewillwaitforamaster'sresponse.Thiswaittimecanbeusedtocontrolafterwhattimethesynchronizationwillgiveupandfail.

ThesyncCollectioncommandmaytakealongtimetocompleteifthecollectionisbig.Theshellwillblockuntiltheslavehassynchronizedtheentirecollectionfromthemasteroruntilanerroroccurs.Bydefault,thesyncCollectioncommandintheArangoShellwillpollforastatusupdateevery10seconds.

WhensyncCollectioniscalledfromtheArangoShell,theoptionalasyncattributecanbeusedtostartthesynchronizationasabackgroundprocessontheslave.Ifasyncissettotrue,thecalltosyncCollectionwillreturnalmostinstantlywithanidstring.Usingthisidstring,thestatusofthesyncjobontheslavecanbequeriedusingthegetSyncResultfunctionasfollows:

varid=replication.syncCollection("test",{

username:"myuser",

async:true

ReplicationLimitationsThereplicationinArangoDBhasafewlimitations.SomeoftheselimitationsmayberemovedinlaterversionsofArangoDB:

thereisnofeedbackfromtheslavestothemaster.Ifaslavecannotapplyaneventitgotfromthemaster,themasterwillhaveadifferentstateofdata.Inthiscase,thereplicationapplierontheslavewillstopandreportanerror.Administratorscantheneither"fix"theproblemorre-syncthedatafromthemastertotheslaveandstarttheapplieragain.atthemomentitisassumedthatonlythereplicationapplierexecuteswriteoperationsonaslave.ArangoDBcurrentlydoesnotpreventusersfromcarryingouttheirownwriteoperationsonslaves,thoughthismightleadtoundefinedbehaviorandthereplicationapplierstopping.whenareplicationslaveasksamasterforlogevents,thereplicationmasterwillreturnallwriteoperationsforuser-definedcollections,butitwillexcludewriteoperationsforcertainsystemcollections.Thefollowingcollectionsareexcludedintentionallyfromreplication:_apps,_trx,_replication,_configuration,_jobs,_queues,_sessions,_foxxlogandallstatisticscollections.Writeoperationsforthefollowingsystemcollectionscanbequeriedfromamaster:_aqlfunctions,_graphs,_users.Foxxapplicationsconsistofdatabaseentriesandapplicationscriptsinthefilesystem.ThefilesystempartsofFoxxapplicationsarenottrackedanywhereandthusnotreplicatedincurrentversionsofArangoDB.ToreplicateaFoxxapplication,itisrequiredtocopytheapplicationtotheremoteserverandinstallitthereusingthefoxx-managerutility.masterserversdonotknowwhichslavesareorwillbeconnectedtothem.Allserversinareplicationsetuparecurrentlyonlylooselycoupled.Therecurrentlyisnowayforaclienttoquerywhichserversarepresentinareplication.whennotusingourmesosintegrationfailovermustbehandledbyclientsorclientAPIs.therecurrentlyisonereplicationapplierperArangoDBdatabase.Itisthusnotpossibletohaveaslaveapplyoperationsfrommultiplemastersintothesametargetdatabase.replicationissetuponaper-databaselevel.WhenusingArangoDBwithmultipledatabases,replicationmustbeconfiguredindividuallyforeachdatabase.thereplicationapplierissingle-threaded,butwriteoperationsonthemastermaybeexecutedinparalleliftheyaffectdifferentcollections.Thusthereplicationappliermightnotbeabletocatchupwithaverypowerfulandloadedmaster.replicationisonlysupportedbetweenthetwoArangoDBserversrunningthesameArangoDBversion.ItiscurrentlynotpossibletoreplicatebetweendifferentArangoDBversions.areplicationappliercannotapplydatafromitself.

SynchronousReplicationAtitscoresynchronousreplicationwillreplicatewriteoperationstomultiplehosts.ThisfeatureisonlyavailablewhenoperatingArangoDBinacluster.Wheneveracoordinatorexecutesasychronouslyreplicatedwriteoperationitwillonlybereportedtobesuccessfulifitwascarriedoutonallreplicas.IncontrasttomultimasterreplicationsetupsknownfromothersystemsArangoDB'ssynchronousoperationguaranteesaconsistentstateacrossthecluster.

Implementation

Architectureinsidethecluster

SynchronousreplicationcanbeconfiguredpercollectionviathepropertyreplicationFactor.Synchronousreplicationrequiresaclustertooperate.

WheneveryouspecifyareplicationFactorgreaterthan1whencreatingacollection,synchronousreplicationwillbeactivatedforthiscollection.Theclusterwilldeterminesuitableleadersandfollowersforeveryrequestedshard(numberOfShards)withinthecluster.Whenrequestingdataofashardonlythecurrentleaderwillbeaskedwhereasfollowerswillonlykeeptheircopyinsync.Thisisduetothecurrentimplementationoftransactions.

Usingsynchronousreplicationalonewillguaranteeconsistencyandhighavailabiltyatthecostofreducedperformance:Writerequestswillhaveahigherlatency(duetoeverywrite-requesthavingtobeexecutedonthefollowers)andreadrequestswon'tscaleoutasonlytheleaderisbeingasked.

Inaclustersynchronousreplicationwillbemanagedbythecoordinatorsfortheclient.Thedatawillalwaysbestoredonprimaries.

ThefollowingexamplewillgiveyouanideaofhowsynchronousoperationhasbeenimplementedinArangoDB.

1. Connecttoacoordinatorviaarangosh2. Createacollection

127.0.0.1:8530@_system>db._create("test",{"replicationFactor":2})

3. thecoordinatorwillfigureoutaleaderand1followerandcreate1shard(asthisisthedefault)

4. Insertdata

127.0.0.1:8530@_system>db.test.insert({"replication":"ᚇ"})

5. Thecoordinatorwillwritethedatatotheleader,whichinturnwillreplicateittothefollower.

6. Onlywhenbothweresuccessfultheresultisreportedtobesuccessful

"_id":"test/7987",

"_key":"7987",

"_rev":"7987"

Whenafollowerfails,theleaderwillgiveuponitafter3secondsandproceedwiththeoperation.Assoonasthefollower(orthenetworkconnectiontotheleader)isbackup,thetwowillresynchronizeandsynchronousreplicationisresumed.Thishappensalltransparentlytotheclient.

ThecurrentimplementationofArangoDBdoesnotallowchangingthereplicationFactorlater.Thisissubjecttochange.Inthemeantimetheonlywayistodumpandrestorethecollection.Seethecookbookrecipeaboutmigrating.

Automaticfailover

Whenevertheleaderofashardisfailingandthereisaquerytryingtoaccessdataofthatshardthecoordinatorwillcontinuetryingtocontacttheleaderuntilittimeouts.Theinternalclustersupervisionrunningontheagencywillcheckclusterhealtheveryfewsecondsandwilltakeactionifthereisnoheartbeatfromaserverfor15seconds.Iftheleaderdoesn'tcomebackintimethesupervisionwillreorganizetheclusterbypromotingforeachshardafollowerthatisinsyncwithitsleadertobethenewleader.Fromthenonthecoordinatorswillcontactthenewleader.

Theprocessisbestoutlinedusinganexample:

1. Theleaderofashard(letsnameitDBServer001)isgoingdown.2. Acoordinatorisaskedtoreturnadocument:

127.0.0.1:8530@_system>db.test.document("100069")

3. ThecoordinatordetermineswhichserverisresponsibleforthisdocumentandfindsDBServer001

4. ThecoordinatortriestocontactDBServer001andtimeoutsbecauseitisnotreachable.5. Afterashortwhilethesupervision(runninginparallelontheagency)willseethatheartbeatsfromDBServer001arenotcomingin6. Thesupervisionpromotesoneofthefollowers(sayDBServer002)thatisinsynctobeleaderandmakesDBServer001afollower.7. AsthecoordinatorcontinuestryingtofetchthedocumentitwillseethattheleaderchangedtoDBServer0028. Thecoordinatortriestocontactthenewleader(DBServer002)andreturnstheresult:

"_key":"100069",

"_id":"test/100069",

"_rev":"513",

"replication":"ᚇ"

9. AfterawhilethesupervisiondeclaresDBServer001tobecompletelydead.10. AnewfollowerisdeterminedfromthepoolofDBservers.11. Thenewfollowersyncsitsdatafromtheleaderandorderisrestored.

Pleasenotethattheremaystillbetimeouts.Dependingonwhenexactlytherequesthasbeendone(inregardtothesupervision)anddependingonthetimeneededtoreconfiguretheclusterthecoordinatormightfailwithatimeouterror!

Configuration

Requirements

SynchronousreplicationrequiresanoperationalArangoDBcluster.

Enablingsynchronousreplication

Synchronousreplicationcanbeenabledpercollection.WhencreatingacollectionyoumayspecifythenumberofreplicasusingthereplicationFactorparameter.Thedefaultvalueissetto1whicheffectivelydisablessynchronousreplication.

Example:

127.0.0.1:8530@_system>db._create("test",{"replicationFactor":3})

Intheabovecase,anywriteoperationwillrequire2replicastoreportsuccessfromnowon.

Preparinggrowth

Youmaycreateacollectionwithhigherreplicationfactorthanavailable.Whenadditionaldbserversbecomeavailabletheshardsareautomaticallyreplicatedtothenewlyavailablemachines.

Multiplereplicasofthesameshardcannevercoexistonthesamedbserverinstance.

SatelliteCollectionsSatelliteCollectionsareanEnterpriseonlyfeature.WhendoingJoinsinanArangoDBclusterdatahastoexchangedbetweendifferentservers.

Joinswillbeexecutedonacoordinator.Itwillprepareanexecutionplanandexecuteit.Whenexecutingthecoordinatorwillcontactallshardsofthestartingpointofthejoinandaskfortheirdata.Thedatabaseserverscarryingoutthisoperationwillloadalltheirlocaldataandthenasktheclusterfortheotherpartofthejoin.Thisagainwillbedistributedtoallinvolvedshardsofthisjoinpart.

Insumthisresultsinmuchnetworktrafficandslowresultsdependingoftheamountofdatathathastobesentthroughoutthecluster.

Satellitecollectionsarecollectionsthatareintendedtoaddressthisissue.

Theywillfacilitatethesynchronousreplicationandreplicateallitsdatatoalldatabaseserversthatarepartofthecluster.

ThisenablesthedatabaseserverstoexecutethatpartofanyJoinlocally.

Thisgreatlyimprovesperformanceforsuchjoinsatthecostsofincreasedstoragerequirementsandpoorerwriteperformanceonthisdata.

TocreateasatellitecollectionsetthereplicationFactorofthiscollectionto"satellite".

Usingarangosh:

arangosh>db._create("satellite",{"replicationFactor":"satellite"});

Afullexample

arangosh>varexplain=require("@arangodb/aql/explainer").explain

arangosh>db._create("satellite",{"replicationFactor":"satellite"})

arangosh>db._create("nonsatellite",{numberOfShards:8})

arangosh>db._create("nonsatellite2",{numberOfShards:8})

Let'sanalyseanormaljoinnotinvolvingsatellitecollections:

arangosh>explain("FORdocinnonsatelliteFORdoc2innonsatellite2RETURN1")

Querystring:

FORdocinnonsatelliteFORdoc2innonsatellite2RETURN1

Executionplan:

IdNodeTypeSiteEst.Comment

1SingletonNodeDBS1*ROOT

4CalculationNodeDBS1-LET#2=1/*jsonexpression*//*constassignment*/

2EnumerateCollectionNodeDBS0-FORdocINnonsatellite/*fullcollectionscan*/

12RemoteNodeCOOR0-REMOTE

13GatherNodeCOOR0-GATHER

6ScatterNodeCOOR0-SCATTER

7RemoteNodeDBS0-REMOTE

3EnumerateCollectionNodeDBS0-FORdoc2INnonsatellite2/*fullcollectionscan*/

5ReturnNodeCOOR0-RETURN#2

Indexesused:

Optimizationrulesapplied:

IdRuleName

1move-calculations-up

2scatter-in-cluster

3remove-unnecessary-remote-scatter

Allshardsinvolvedqueryingthenonsatellitecollectionwillfanoutviathecoordinatortotheshardsofnonsatellite.Insum8shardswillopen8connectionstothecoordinatoraskingfortheresultsofthenonsatellite2join.Thecoordinatorwillfanouttothe8shardsofnonsatellite2.Sotherewillbequitesomenetworktraffic.

Let'snowhavealookatthesameusingsatellitecollections:

arangosh>db._query("FORdocinnonsatelliteFORdoc2insatelliteRETURN1")

Querystring:

FORdocinnonsatelliteFORdoc2insatelliteRETURN1

Executionplan:

IdNodeTypeSiteEst.Comment

1SingletonNodeDBS1*ROOT

4CalculationNodeDBS1-LET#2=1/*jsonexpression*//*constassignment*/

2EnumerateCollectionNodeDBS0-FORdocINnonsatellite/*fullcollectionscan*/

3EnumerateCollectionNodeDBS0-FORdoc2INsatellite/*fullcollectionscan,satellite*/

5ReturnNodeCOOR0-RETURN#2

Indexesused:

Optimizationrulesapplied:

IdRuleName

1move-calculations-up

2scatter-in-cluster

3remove-unnecessary-remote-scatter

4remove-satellite-joins

Inthisscenarioallshardsofnonsatellitewillbecontacted.Howeverasthejoinisasatellitejoinallshardscandothejoinlocallyasthedataisreplicatedtoallserversreducingthenetworkoverheaddramatically.

Caveats

Theclusterwillautomaticallykeepallsatellitecollectionsonallserversinsyncbyfacilitatingthesynchronousreplication.Thismeansthatwritewillbeexecutedontheleaderonlyandthisserverwillcoordinatereplicationtothefollowers.Ifafollowerdoesn'tanswerintime(duetonetworkproblems,temporaryshutdownetc.)itmayberemovedasafollower.ThisisbeingreportedtotheAgency.

Thefollower(oncebackinbusiness)willthenperiodicallychecktheAgencyandknowthatitisoutofsync.Itwillthenautomaticallycatchup.Thismaytakeawhiledependingonhowmuchdatahastobesynced.WhendoingajoininvolvingthesatelliteyoucanspecifyhowlongtheDBServerisallowedtowaitforsyncuntilthequeryisbeingaborted.

CheckAccessingCursorsfordetails.

DuringnetworkfailurethereisalsoaminimalchancethataquerywasproperlydistributedtotheDBServersbutthataprevioussatellitewritecouldnotbereplicatedtoafollowerandtheleaderdroppedthefollower.Thefollowerhoweveronlycheckseveryfewsecondsifitisreallyinsyncsoitmightindeeddeliverstaleresults.

DatacentertodatacenterreplicationadministrationThisSectionincludesinformationrelatedtotheadministrationofthedatacentertodatacenterreplication.

Forageneralintroductiontothedatacentertodatacenterreplication,pleaserefertotheDatacentertodatacenterreplicationchapther.

Startingsynchronization

OnceallcomponentsoftheArangoSyncsolutionhavebeendeployedandarerunningproperly,ArangoSyncwillnotautomaticallyreplicatedatabasestructureandcontent.Forthat,itisisneededtoconfiguresynchronization.

Toconfiguresynchronization,youneedthefollowing:

Theendpointofthesyncmasterinthetargetdatacenter.Theendpointofthesyncmasterinthesourcedatacenter.Acertificate(inkeyfileformat)usedforclientauthenticationofthesyncmaster(withthesyncmasterinthesourcedatacenter).ACAcertificate(publickeyonly)forverifyingtheintegrityofthesyncmasters.Ausername+passwordpair(orclientcertificate)forauthenticatingtheconfigurerequirewiththesyncmaster(inthetargetdatacenter)

Withthatinformation,run:

arangosyncconfiguresync\

--master.endpoint=<endpointsofsyncmastersintargetdatacenter>\

--master.keyfile=<keyfileofofsyncmastersintargetdatacenter>\

--source.endpoint=<endpointsofsyncmastersinsourcedatacenter>\

--source.cacert=<publickeyofCAcertificateusedtoverifysyncmasterinsourcedatacenter>\

--auth.user=<usernameusedforauthenticationofthiscommand>\

--auth.password=<passwordofauth.user>

Thecommandwillfinishquickly.Afterwardsitwilltakesometimeuntiltheclustersinbothdatacentersareinsync.

Usethefollowingcommandtoinspectthestatusofthesynchronizationofadatacenter:

arangosyncgetstatus\

--master.endpoint=<endpointsofsyncmastersindatacenterofinterest>\

--auth.password=<passwordofauth.user>\

Note:Invokingthiscommandonthetargetdatacenterwillreturndifferentresultsfrominvokingitonthesourcedatacenter.Youneedinsightinbothresultstogeta"completepicture".

Wherethegetstatuscommandgivesinsightinthestatusofsynchronization,therearemoredetailedcommandstogiveinsightintasks&registeredworkers.

Usethefollowingcommandtogetalistofallsynchronizationtasksinadatacenter:

arangosyncgettasks\

Usethefollowingcommandtogetalistofallmastersinadatacenterandknowwhichmasteristhecurrentleader:

arangosyncgetmasters\

Usethefollowingcommandtogetalistofallworkersinadatacenter:

arangosyncgetworkers\

Stopingsynchronization

Ifyounolongerwanttosynchronizedatafromasourcetoatargetdatacenteryoumuststopit.Todoso,runthefollowingcommand:

arangosyncstopsync\

Thecommandwillwaituntilsynchronizationhascompletelystoppedbeforereturning.Ifthesynchronizationisnotcompletelystoppedwithinareasonableperiod(2minutesbydefault)thecommandwillfail.

Ifthesourcedatacenterisnolongeravailableitisnotpossibletostopsynchronizationinagracefulmanner.Ifthathappensabortthesynchronizationwiththefollowingcommand:

arangosyncabortsync\

Ifthesourcedatacenterrecoversafteranabortsynchasbeenexecuted,itisneededto"cleanup"ArangoSyncinthesourcedatacenter.Todoso,executethefollowingcommand:

arangosyncabortoutgoingsync\

--master.endpoint=<endpointsofsyncmastersinsourcedatacenter>\

ReversingsynchronizationdirectionIfyouwanttoreversethedirectionofsynchronization(e.g.afterafailureindatacenterAandyouswitchedtothedatacenterBforfallback),youmustfirststop(orabort)theoriginalsynchronization.

Oncethatisfinished(andcleanuphasbeenappliedincaseofabort),youmustnowconfigurethesynchronizationagain,butwithswappedsource&targetsettings.

ShardingArangoDBisorganizingitscollectiondatainshards.ShardingallowstousemultiplemachinestorunaclusterofArangoDBinstancesthattogetherconstituteasingledatabase.Thisenablesyoutostoremuchmoredata,sinceArangoDBdistributesthedataautomaticallytothedifferentservers.Inmanysituationsonecanalsoreapabenefitindatathroughput,againbecausetheloadcanbedistributedtomultiplemachines.

Shardsareconfiguredpercollectionsomultipleshardsofdataformthecollectionasawhole.TodetermineinwhichshardthedataistobestoredArangoDBperformsahashacrossthevalues.Bydefaultthishashisbeingcreatedfrom_key.

Toconfigurethenumberofshards:

127.0.0.1:8529@_system>db._create("sharded_collection",{"numberOfShards":4});

Toconfigurethehashingforanotherattribute:

127.0.0.1:8529@_system>db._create("sharded_collection",{"numberOfShards":4,"shardKeys":["country"]});

Thiswouldbeusefultokeepdataofeverycountryinoneshardwhichwouldresultinbetterperformanceforqueriesworkingonapercountrybase.YoucanalsospecifymultipleshardKeys.Notehoweverthatifyouchangetheshardkeysfromtheirdefault["_key"],thenfindingadocumentinthecollectionbyitsprimarykeyinvolvesarequesttoeverysingleshard.Furthermore,inthiscaseonecannolongerprescribetheprimarykeyvalueofanewdocumentbutmustusetheautomaticallygeneratedone.Thislatterrestrictioncomesfromthefactthatensuringuniquenessoftheprimarykeywouldbeveryinefficientiftheusercouldspecifytheprimarykey.

Onwhichnodeinaclusteraparticularshardiskeptisundefined.Thereisnooptiontoconfigureanaffinitybasedoncertainshardkeys.

Uniqueindexes(hash,skiplist,persistent)onshardedcollectionsareonlyallowedifthefieldsusedtodeterminetheshardkeyarealsoincludedinthelistofattributepathsfortheindex:

shardKeys indexKeys

a a ok

a b notok

a a,b ok

a,b a notok

a,b b notok

a,b a,b ok

a,b a,b,c ok

a,b,c a,b notok

a,b,c a,b,c ok

Sharding

GeneralUpgradeInformation

Recommendedmajorupgradeprocedure

ToupgradeanexistingArangoDB2.xto3.0pleaseusetheproceduredescribedhere.

Recommendedminorupgradeprocedure

ToupgradeanexistingArangoDBdatabasetoanewerversionofArangoDB(e.g.3.0to3.1,or3.1to3.2),thefollowingmethodisrecommended:

ChecktheCHANGELOGandthelistofincompatiblechangesforAPIorotherchangesinthenewversionofArangoDBandmakesureyourapplicationscandealwiththemStopthe"old"arangodserviceorbinaryCopytheentire"old"datadirectorytoasafeplace(thatis,abackup)InstallthenewversionofArangoDBandstarttheserverwiththe--database.auto-upgradeoptiononce.ThismightwritetothelogfileofArangoDB,soyoumaywanttocheckthelogsforanyissuesbeforegoingon.Startthe"new"arangodserviceorbinaryregularlyandcheckthelogsforanyissues.Whenyou'reconfidenteverythingwentwell,youmaywanttocheckthedatabasedirectoryforanyfileswiththeending.old.ThesefilesarecreatedbyArangoDBduringupgradesandcanbesafelyremovedmanuallylater.

Ifanythinggoeswrongduringorshortlyaftertheupgrade:

Stopthe"new"arangodserviceorbinaryReverttothe"old"arangodbinaryandrestorethe"old"datadirectoryStartthe"old"versionagain

ItisnotsupportedtousedatafilescreatedormodifiedbyanewerversionofArangoDBwithanolderArangoDBversion.Forexample,itisunsupportedandislikelytocauseproblemswhenusing3.2datafileswithanArangoDB3.0instance.

Switchingthestorageengine

Inordertouseadifferentstorageenginewithanexistingdatadirectory,itisrequiredtofirstcreatealogicalbackupofthedatausingarangodump.

Afterthat,thearangodservershouldberestartedwiththedesiredstorageengineselected(thiscanbedonebysettingtheoption--server.storage-engine)andusinganon-existingdatadirectory.

Whentheserverisupandrunningwiththedesiredstorageengine,thedatacanbere-importedusingarangorestore.

Upgrading

UpgradingtoArangoDB3.3PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB3.3.Pleasebesurethatyouhavecheckedthelistofchangesin3.3beforeupgrading.

Upgradingto3.3

Switchingthestorageengine

Inordertouseadifferentstorageenginewithanexistingdatadirectory,itisrequiredtofirstcreatealogicalbackupofthedatausingarangodump.Thatbackupshouldbecreatedbeforetheupgradeto3.2.

Afterthat,theArangoDBinstallationcanbeupgradedandstopped.Theservershouldthenberestartedwiththedesiredstorageengineselected(thiscanbedonebysettingtheoption--server.storage-engine)andusinganon-existingdatadirectory.Thiswillstarttheserverwiththeselectedstorageenginebutwithnodata.

Whentheserverisupandrunning,thedatafromthelogicalbackupcanbere-importedusingarangorestore.

Upgradingto3.2

Upgradingto3.1

MigratingdatabasesandcollectionsfromArangoDB2.8to3.0

ArangoDB3.0doesnotprovideanautomaticupdatemechanismfordatabasedirectoriescreatedwiththe2.xbranchesofArangoDB.

InordertomigratedatafromArangoDB2.8(oranolder2.xversion)intoArangoDB3.0,itisnecessarytoexportthedatafrom2.8usingarangodump,andthenimportthedumpintoafreshArangoDB3.0witharangorestore.

Todothis,firstrunthe2.8versionofarangodumptoexportthedatabasedataintoadirectory.arangodumpwilldumpthe_systemdatabasebydefault.Inordertomakeitdumpmultipledatabases,itneedstobeinvokedoncepersourcedatabase,e.g.

#in2.8

arangodump--server.database_system--output-directorydump-system

arangodump--server.databasemydb--output-directorydump-mydb

Thatwillproduceadumpdirectoryforeachdatabasethatarangodumpiscalledfor.Iftheserverhasauthenticationturnedon,itmaybenecessarytoprovidetherequiredcredentialswheninvokingarangodump,e.g.

arangodump--server.database_system--server.usernamemyuser--server.passwordmypasswd--output-directorydump-system

ThedumpsproducedbyarangodumpcannowbeimportedintoArangoDB3.0usingthe3.0versionofarangodump:

#in3.0

arangorestore--server.database_system--input-directorydump-system

arangorestore--server.databasemydb--input-directorydump-mydb

arangorestorewillbydefaultfailifthetargetdatabasedoesnotexist.Itcanbetoldtocreateitautomaticallyusingtheoption--create-databasetrue:

arangorestore--server.databasemydb--create-databasetrue--input-directorydump-mydb

Andagainitmayberequiredtoprovideaccesscredentialswheninvokingarangorestore:

arangorestore--server.databasemydb--create-databasetrue--server.usernamemyuser--server.passwordmypasswd--input-directo

rydump-system

Pleasenotethattheversionofdump/restoreshouldmatchtheserverversion,i.e.itisrequiredtodumptheoriginaldatawiththe2.8versionofarangodumpandrestoreitwiththe3.0versionofarangorestore.

Afterthatthe3.0instanceofArangoDBwillcontainthedatabasesandcollectionsthatwerepresentinthe2.8instance.

Adjustingauthenticationinfo

AuthenticationinformationwasstoredperdatabaseinArangoDB2.8,meaningtherecouldbedifferentusersandaccesscredentialsperdatabase.In3.0,theusersarestoredinacentrallocationinthe_systemdatabase.Tousethesameusersetupasin2.8,itmayberequiredtocreateextrausersand/oradjusttheirpermissions.

Inordertodothat,pleaseconnecttothe3.0instancewithanArangoShell(thiswillconnecttothe_systemdatabasebydefault):

arangosh--server.usernamemyuser--server.passwordmypasswd

Upgradingto3.0

Usethefollowingcommandstocreateanewuserwithsomepasswordandgrantthemaccesstoaspecificdatabase

require("@arangodb/users").save(username,password,true);

require("@arangodb/users").grantDatabase(username,databaseName,"rw");

Forexample,tocreateausermyuserwithpasswordmypasswdandgivethemaccesstodatabasesmydb1andmydb2,thecommandswouldlookasfollows:

require("@arangodb/users").save("myuser","mypasswd",true);

require("@arangodb/users").grantDatabase("myuser","mydb1","rw");

require("@arangodb/users").grantDatabase("myuser","mydb2","rw");

Existinguserscanalsobeupdated,removedorlistedusingthefollowingcommands:

/*updateusermyuserwithpasswordmypasswd*/

require("@arangodb/users").update("myuser","mypasswd",true);

/*removeusermyuser*/

require("@arangodb/users").remove("myuser");

/*listallusers*/

require("@arangodb/users").all();

Foxxapplications

Thedump/restoreproceduredescribedabovewillnotexportandre-importFoxxapplications.Inordertomovethesefrom2.8to3.0,Foxxapplicationsshouldbeexportedaszipfilesviathe2.8webinterface.

Thezipfilescanthenbeuploadedinthe"Services"sectionintheArangoDB3.0webinterface.Applicationsmayneedtobeadjustedmanuallytorunin3.0.PleaseconsultthemigrationguideforFoxxapps.

AnalternativewayofmovingFoxxappsinto3.0istocopythesourcedirectoryofa2.8Foxxapplicationmanuallyintothe3.0Foxxappsdirectoryforthetargetdatabase(whichisnormally/var/lib/arangodb3-apps/_db/<dbname>/buttheexactlocationisplatform-specific).

Upgradingto3.0

PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.8cannotbeusedwithearlierversions(e.g.ArangoDB2.7)anymore.Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDBinstallationbeforeperforminganupgrade.

DatabaseDirectoryVersionCheckandUpgrade

ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.8encountersadatabasecreatedwithearlierversionsofArangoDB,itwillrefusetostart.Thisisintentional.

Theoutputwillthenlooklikethis:

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Databasedirectoryversion(20702)islowerthancurrentversion(208

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':---------------------------------------------------------------------

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':--upgrade

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbstop

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade

2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbstart

2015-12-04T17:11:17Z[31432]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption

TomakeArangoDB2.8startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption.

Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe--upgradecommandoneachindividualdatabasesubfolder(nameddatabase-<somenumber>).

Forexample,ifyouregularlystartyourArangoDBserverwith

unix>arangodmydatabasefolder

thenrunning

unix>arangodmydatabasefolder--upgrade

willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.

Startingwith--upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackupofyourdatabasedirectorybeforeperformingtheupgrade.

Thelastlineoftheoutputshouldlooklikethis:

2015-12-04T17:12:15Z[31558]INFOdatabaseupgradepassed

Pleasecheckthefulloutputthe--upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.8regularly.

Upgradingaclusterplannedinthewebinterface

Upgradingto2.8

AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellasrunningtheversioncheckonthewholerunningclusterintheend.

Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministratedaclusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:

1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.

2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.

3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchtheclusterinthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillseeanotherbuttonmarked"Upgradeandrelaunchcluster".

4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillseetheusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDBinstances.Pleaseletusknowifyourunintoproblems.

ThereisanalternativewayusingtheArangoDBshell.Insteadofsteps3.and4.aboveyoucanlauncharangosh,pointittothedispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption--server.endpoint,andexecute

arangosh>require("org/arangodb/cluster").Upgrade("root","");

Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace"root"withausernameand""withapasswordthatisvalidforauthenticationwiththecluster.

UpgradingFoxxappsgeneratedbyArangoDB2.7andearlier

TheimplementationoftherequirefunctionusedtoimportmodulesinArangoDBandFoxxhaschangedinordertoimprovecompatibilitywithNode.jsmodules.

Givenanapp/servicewiththefollowinglayout:

manifest.jsoncontrollers/

todos.jsmodels/

todo.jsrepositories/

todos.jsnode_modules/

models/todo.js

Thefilecontrollers/todos.jswouldpreviouslycontainthefollowingrequirecalls:

var_=require('underscore');

varjoi=require('joi');

varFoxx=require('org/arangodb/foxx');

varArangoError=require('org/arangodb').ArangoError;

varTodos=require('repositories/todos');//<--!

varTodo=require('models/todo');//<--!

Therequirepathsrepositories/todosandmodels/todowerepreviouslyresolvedlocallyasrelativetotheapproot.

Startingwith2.8thesepathswouldinsteadberesolvedasrelativetothenode_modulesfolderortheglobalArangoDBmodulepathsbeforebeingresolvedlocallyasafallback.

Inthegivenexamplelayouttheappwouldbreakin2.8becausethemodulenamemodels/todowouldalwaysresolvetonode_modules/models/todo.js(whichpreviouslywouldhavebeenignored)insteadofthelocalmodels/todo.js.

Inordertomakesuretheappstillworksin2.8,therequirecallsincontrollers/todos.jswouldneedtobeadjustedtolooklikethis:

Upgradingto2.8

var_=require('underscore');

varjoi=require('joi');

varArangoError=require('org/arangodb').ArangoError;

varTodos=require('../repositories/todos');//<--!

varTodo=require('../models/todo');//<--!

Notethattheold"global"stylerequirecallsmaystillworkin2.8butmaybreakunexpectedlyifmoduleswithmatchingnamesareinstalledglobally.

Upgradingto2.8

2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':----------------------------------------------------------------------

TomakeArangoDB2.6startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption.

thenrunning

Upgradingto2.6

In2.5wehavealsochangedthepathsforFoxxapplications.PleasealsomakesurethatyouhaveabackupofallFoxxappsinyourjavascript.app-pathandjavascript.dev-app-path.ItissufficienttohavethesourcefilesforFoxxsomewhereelsesoyoucanreinstallthemonerror.Tocheckthateverythinghasworkedduringupgradeyoucouldusetheweb-interfaceApplicationstabor

unix>foxx-managerlist

forallyourdatabases.Thelistedappsshouldbeidenticalbeforeandaftertheupgrade.

TomakeArangoDB2.5startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption.Note:WehavechangedFoxxfolderstructureandimplementedanupgradetasktomoveyourapplicationstothenewstructure.InordertotellthisupgradetasktoalsomoveyourdevelopmentFoxxappspleasemakesureyougivethedev-app-pathaswell.IfyouhavenotuseddevelopmentmodeforFoxxappsyoucandropthe--javascript.dev-app-path.Itisonlypossibletoupgradeonedev-app-pathtogetherwithonedatafolder.

unix>arangoddata--upgrade--javascript.dev-app-pathdevapps

wheredataisArangoDB'smaindatadirectoryanddevappsisthedirectorywhereyoudevelopFoxxapps.

thenrunning

Upgradingto2.5

UpgradingaclusterplannedinthewebinterfaceAclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellasrunningtheversioncheckonthewholerunningclusterintheend.

Upgradingto2.5

TomakeArangoDB2.4startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption:

unix>arangoddata--upgrade

wheredataisArangoDB'smaindatadirectory.

thenrunning

Upgradingto2.4

2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':Databasedirectoryversion(2.2)islowerthancurrentversion(20300)

thenrunning

Theoutputshouldlooklikethis:

2014-11-03T15:48:47Z[2708]INFOIndatabase'_system':Found24definedtask(s),5task(s)torun

2014-11-03T15:48:47Z[2708]INFOIndatabase'_system':stateprod/standalone/upgrade,tasksupdateUserModel,createStatistics,

upgradeClusterPlan,setupQueues,setupJobs

2014-11-03T15:48:48Z[2708]INFOIndatabase'_system':upgradesuccessfullyfinished

Upgradingto2.3

Pleasechecktheoutputthe--upgraderun.Itmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifnoerrorsarepresentortheyhavebeenresolved,youcanstartArangoDB2.3regularly.

Upgradingto2.3

UpgradingtoArangoDB2.2PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.2.

2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':Databasedirectoryversion(2.1)islowerthanserverversion(2.2).

2014-07-07T22:04:53Z[18675]FATALDatabaseversioncheckfailedfor'_system'.Pleasestarttheserverwiththe--upgradeopti

thenrunning

Theoutputshouldlooklikethis:

2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':startingupgradefromversion2.1to2.2.0

2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':Found19definedtask(s),2task(s)torun

2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':upgradesuccessfullyfinished

Upgradingto2.2

Pleasechecktheoutputthe--upgraderun.Itmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifnoerrorsarepresentortheyhavebeenresolved,youcanstartArangoDB2.2regularly.

Upgradingto2.2

Troubleshooting

ArangodIftheArangoDBserverdoesnotstartorifyoucannotconnecttoitusingarangoshorotherclients,youcantrytofindtheproblemcausebyexecutingthefollowingsteps.Iftheserverstartsupwithoutproblemsyoucanskipthissection.

Checktheserverlogfile:Iftheserverhaswrittenalogfileyoushouldcheckitbecauseitmightcontainrelevanterrorcontextinformation.

Checktheconfiguration:Theserverlooksforaconfigurationfilenamedarangod.confonstartup.Thecontentsofthisfilewillbeusedasabaseconfigurationthatcanoptionallybeoverriddenwithcommand-lineconfigurationparameters.Youshouldchecktheconfigfileforthemostrelevantparameterssuchas:

server.endpoint:WhatIPaddressandporttobindtologparameters:Ifandwheretologdatabase.directory:Paththedatabasefilesarestoredin

Iftheconfigurationrevealsthatsomethingisnotconfiguredrighttheconfigfileshouldbeadjustedandtheserverberestarted.

Starttheservermanuallyandcheckitsoutput:Startingtheservermightfailevenbeforeloggingisactivatedsotheserverwillnotproducelogoutput.Thiscanhappeniftheserverisconfiguredtowritethelogstoafilethattheserverhasnopermissionson.Inthiscasetheservercannotloganerrortothespecifiedlogfilebutwillwriteastartuperroronstderrinstead.Startingtheservermanuallywillalsoallowyoutooverridespecificconfigurationoptions,e.g.toturnon/offfileorscreenloggingetc.

ChecktheTCPport:Iftheserverstartsupbutdoesnotacceptanyincomingconnectionsthismightbeduetofirewallconfigurationbetweentheserverandanyclient(s).TheserverbydefaultwilllistenonTCPport8529.PleasemakesurethisportisactuallyaccessiblebyotherclientsifyouplantouseArangoDBinanetworksetup.

Whenusinghostnamesintheconfigurationorwhenconnecting,pleasemakesurethehostnameisactuallyresolvable.ResolvinghostnamesmightinvokeDNS,whichcanbeasourceoferrorsonitsown.

ItisgenerallygoodadvicetonotuseDNSwhenspecifyingtheendpointsandconnectionaddresses.UsingIPaddressesinsteadwillruleoutDNSasasourceoferrors.Anotheralternativeistouseahostnamespecifiedinthelocal/etc/hostsfile,whichwillthenbypassDNS.

Testifcurlcanconnect:Oncetheserverisstarted,youcanquicklyverifyifitrespondstorequestsatall.Thischeckallowsyoutodeterminewhetherconnectionerrorsareclient-specificornot.Ifatleastoneclientcanconnect,itislikelythatconnectionproblemsofotherclientsarenotduetoArangoDB'sconfigurationbutduetoclientorin-betweennetworkconfigurations.

Youcantestconnectivityusingasimplecommandsuchas:

curl--dump--XGEThttp://127.0.0.1:8529/_api/version&&echo

ThisshouldreturnaresponsewithanHTTP200statuscodewhentheserverisrunning.Ifitdoesitalsomeanstheserverisgenerallyacceptingconnections.Alternativetoolstocheckconnectivityarelynxorab.

arangod

EmergencyConsoleTheArangoDBdatabaseserverhastwomodesofoperation:Asaserver,whereitwillanswertoclientrequestsandasanemergencyconsole,inwhichyoucanaccessthedatabasedirectly.Thelatter-asthenamesuggests-shouldonlybeusedincaseofanemergency,forexample,acorruptedcollection.Usingtheemergencyconsoleallowsyoutoissueallcommandsnormallyavailableinactionsandtransactions.Whenstartingtheserverinemergencyconsolemode,theservercannothandleanyclientrequests.

Youshouldneverstartmorethanoneserverusingthesamedatabasedirectory,independentofthemodeofoperation.Normally,ArangoDBwillpreventyoufromdoingthisbyplacingalockfileinthedatabasedirectoryandnotallowingasecondArangoDBinstancetousethesamedatabasedirectoryifalockfileisalreadypresent.

InCaseOfDisaster

Thefollowingcommandstartsanemergencyconsole.

Note:Neverstarttheemergencyconsoleforadatabasewhichalsohasaserverattachedtoit.Ingeneral,theArangoDBshelliswhatyouwant.

>./arangod--console--logerror/tmp/vocbase

ArangoDBshell[V8version5.0.71.39,DBversion3.x.x]

arango>1+2;

arango>vardb=require("@arangodb").db;db.geo.count();

TheemergencyconsoleprovidesaJavaScriptconsoledirectlyrunninginthearangodserverprocess.ThisallowstodebugandexaminecollectionsanddocumentsaswiththenormalArangoDBshell,butwithoutclient/servercommunication.

However,itisverylikelythatyouwillneverneedtheemergencyconsoleunlessyouareanArangoDBdeveloper.

EmergencyConsole

DatafileDebugger

InCaseOfDisaster

AranagoDBusesappend-onlyjournals.Datacorruptionshouldonlyoccurwhenthedatabaseserveriskilled.Inthiscase,thecorruptionshouldonlyoccurinthelastobject(s)thathavebeingwrittentothejournal.

Ifacorruptionoccurswithinanormaldatafile,thenthiscanonlyhappenifahardwarefaultoccurred.

Ifajournalordatafileiscorrupt,shutdownthedatabaseserverandstarttheprogram

unix>arango-dfdb

inordertochecktheconsistencyofthedatafilesandjournals.Thisbringsup

_________________

/\___||____/_(_)|___/\/__\/_\

//\/_`|__/_`||_|||/_\//\/__\////_\/

//_//(_||||(_||_|||__///_//\/\//_\\

/___,'\__,_|\__\__,_|_||_|_|\___|/___,'\_____/\____/

Availablecollections:

0:_structures

1:_users

2:_routing

3:_modules

4:_graphs

5:products

6:prices

Collectiontocheck:

Youcannowselectwhichdatabaseandcollectionyouwanttocheck.Afteryouselectedoneorallofthecollections,aconsistencycheckwillbeperformed.

Checkingcollection#1:_users

Database

path:/usr/local/var/lib/arangodb

Collection

name:_users

identifier:82343

Datafiles

#ofjournals:1

#ofcompactors:1

#ofdatafiles:0

Datafile

path:/usr/local/var/lib/arangodb/collection-82343/journal-1065383.db

type:journal

currentsize:33554432

maximalsize:33554432

totalused:256

#ofentries:3

status:OK

Ifthereisaproblemwithoneofthedatafiles,thenthedatabasedebuggerwillprintitandpromptforwhethertoattempttofixit.

WARNING:Thejournalwasnotclosedproperly,thelastentriesarecorrupted.

ThismighthappenArangoDBwaskilledandthelastentrieswerenot

fullywrittentodisk.

DatafileDebugger

Wipethelastentries(Y/N)?

IfyouanswerY,thecorruptedentrywillberemoved.

Ifyouseeacorruptioninadatafile(andnotajournal),thensomethingisterriblywrong.ThesefilesareimmutableandneverchangedbyArangoDB.Acorruptioninsuchfileisanindicationofahard-diskfailure.

DatafileDebugger

ArangobenchArangobenchisArangoDB'sbenchmarkandtesttool.Itcanbeusedtoissuetestrequeststothedatabaseforperformanceandserverfunctiontesting.Itsupportsparallelqueryingandbatchrequests.

Relatedblogposts:

MeasuringArangoDBinsertperformanceGainfactorof5usingbatchrequests

Startupoptions--async:Sendasynchronousrequests.Thedefaultvalueisfalse.

--batch-size:Numberofoperationstosendperbatch.Use0todisablebatching(thisisthedefault).

--collection:Nameofcollectiontouseintest(onlyrelevantforteststhatinvokecollections).

--replication-factor:Incaseofacluster,thereplicationfactorofthecreatedcollections.

--number-of-shards:Incaseofacluster,thenumberofshardsofthecreatedcollections.

--wait-for-sync:ThevalueofwaitForSyncforcreatedcollections.

--complexity:Complexityvaluefortestcase(default:1).Meaningdependsontestcase.

--concurrency:Numberofparallelthreadsthatwillissuerequests(default:1).

--configuration:Readconfigurationfromfile.

--delay:Useastartupdelay.Thisisonlynecessarywhenruninseries.Thedefaultvalueisfalse.

--keep-alive:UseHTTPkeep-alive(default:true).

--progress:Showprogressofbenchmark,onevery20threquest.Settofalsetodisableintermediatelogging.Thedefaultvalueistrue.

--requests:Totalnumberofrequeststoperform(default:1000).

--server.endpoint:Serverendpointtoconnectto,consistingofprotocol,IPaddressandport.Defaultstotcp://localhost:8529.

--server.database:Databasenametousewhenconnecting(default:"_system").

--server.username:Usernametousewhenconnecting(default:"root").

--server.password:Passwordtousewhenconnecting.Don'tspecifythisoptiontogetapasswordprompt.

--server.authentication:Wetherornottoshowthepasswordpromptanduseauthenticationwhenconnectingtotheserver(default:true).

--test-case:Nameoftestcasetoperform(default:"version").Possiblevalues:

version:requests/_api/versiondocument:createsdocumentscollection:createscollectionsimport-document:createsdocumentsviatheimportAPIhash:Create/Read/Update/Readdocumentsindexedbyahashindexskiplist:Create/Read/Update/Readdocumentsindexedbyaskiplistedge:Create/Read/Updateedgedocumentsshapes:Create&Deletedocumentswithheterogeneousattributenamesshapes-append:Createdocumentswithheterogeneousattributenamesrandom-shapes:Create/Read/Deleteheterogeneousdocumentswithrandomvaluescrud:Create/Read/Update/Deletecrud-append:Create/Read/Update/Readagain

Arangobench

crud-write-read:Create/ReadDocumentsaqltrx:AQLTransactionswithdeepnestedAQLFOR-loopscounttrx:usesJStransactionstocountthedocumentsandinserttheresultagainmultitrx:multipletransactionscombiningreads&writesfromjsmulti-collection:multipletransactionscombiningreads&writesfromjsonmultiplecollectionsaqlinsert:insertdocumentsviaAQLaqlv8:executeAQLwithV8functionstoinsertrandomdocuments

--verbose:PrintoutrepliesiftheHTTPheaderindicatesDBerrors.(default:false).

Examples

arangobench

StartsArangobenchwiththedefaultuserandserverendpoint.

--test-caseversion--requests1000--concurrency1

Runsthe'version'testcasewith1000requests,withoutconcurrency.

--test-casedocument--requests1000--concurrency2

Runsthe'document'testcasewith2000requests,withtwoconcurrentthreads.

--test-casedocument--requests1000--concurrency2--asynctrue

Runsthe'document'testcasewith2000requests,withconcurrency2,withasyncrequests.

--test-casedocument--requests1000--concurrency2--batch-size10

Runsthe'document'testcasewith2000requests,withconcurrency2,usingbatchrequests.

Arangobench

TroubleshootingdatacentertodatacenterreplicationThedatacentertodatacenterreplicationisadistributedsystemwithalotdifferentcomponents.Aswithanysuchsystem,itrequiressome,butnotalot,ofoperationalsupport.

Thissectionincludesinformationonhowtotroubleshootthedatacentertodatacenterreplication.

Forageneralintroductiontothedatacentertodatacenterreplication,pleaserefertotheDatacentertodatacenterreplicationchapter.

WhatmeansareavailabletomonitorstatusAllofthecomponentsofArangoSyncprovidemeanstomonitortheirstatus.Belowyou'llfindanoverviewpercomponent.

Syncmaster&workers:Thearangosyncserversrunningaseithermasterorworker,provide:

AstatusAPI,seearangosyncgetstatus.Makesurethatallstatusesreportrunning.Forevenmoredetailthefollowingcommandsarealsoavailable:arangosyncgettasks,arangosyncgetmasters&arangosyncgetworkers.Alogonthestandardoutput.Loglevelscanbeconfiguredusing--log.levelsettings.AmetricsAPIGET/metrics.ThisAPIiscompatiblewithPrometheus.SampleGrafanadashboardsforinspectingthesemetricsareavailable.

ArangoDBcluster:ThearangodserversthatmakeuptheArangoDBclusterprovide:

Alogfile.Thisisconfigurablewithsettingswithalog.prefix.E.g.--log.output=file://myLogFileor--log.level=info.AstatisticsAPIGET/_admin/statistics

Kafkacluster:Thekafkabrokersprovide:

Alogfile,seesettingswithlog.prefixinitsserver.propertiesconfigurationfile.Zookeeper:Thezookeeperagentsprovide:

Alogonstandardoutput.

WhattolookforwhilemonitoringstatusTheveryfirstthingtodowhenmonitoringthestatusofArangoSyncistolookintothestatusprovidedbyarangosyncgetstatus...-v.Whennoteverythingisintherunningstate(onbothdatacenters),thisisanindicationthatsomethingmaybewrong.Incasethathappens,giveitsometime(incrementalsynchronizationmaytakequitesometimeforlargecollections)andlookatthestatusagain.Ifthestatusesdonotchange(orchange,butnotreachrunning)itistimetoinspectsthemetrics&logfiles.Whenthemetricsorlogsseemtoindicateaprobleminasyncmasterorworker,itissafetorestartit,aslongasonly1instanceisrestartedatatime.Giverestartedinstancessometimeto"catchup".

WhattodowhenproblemsremainWhenaproblemremainsandrestartingmasters/workersdoesnotsolvetheproblem,contactsupport.Makesuretoincludeprovidesupportwiththefollowinginformation:

Outputofarangosyncgetversion...onbothdatacenters.Outputofarangosyncgetstatus...-vonbothdatacenters.Outputofarangosyncgettasks...-vonbothdatacenters.Outputofarangosyncgetmasters...-vonbothdatacenters.Outputofarangosyncgetworkers...-vonbothdatacenters.LogfilesofallcomponentsAcompletedescriptionoftheproblemyouobservedandwhatyoudidtoresolveit.

HowtomonitorstatusofArangoSync

Howtokeepitalive

Whattodoincaseoffailuresorbugs

Whattodowhenasourcedatacenterisdown

WhenyouuseArangoSyncforbackupofyourclusterfromonedatacentertoanotherandthesourcedatacenterhasacompleteoutage,youmayconsiderswitchingyourapplicationstothetarget(backup)datacenter.

Thisiswhatyoumustdointhatcase:

1. Stopsynchronizationusing:

arangosyncstopsync...

Whenthesourcedatacenteriscompletelyunresponsivethiswillnotsucceed.Inthatcaseuse:

arangosyncabortsync...

SeeStopingsynchronizationforhowtocleanupthesourcedatacenterwhenitbecomesavailableagain.

2. Verifythatconfigurationhascompletelystoppedusing:

arangosyncgetstatus...-v

3. Reconfigureyourapplicationstousethetarget(backup)datacenter.

Whentheoriginalsourcedatacenterisrestored,youmayswitchrolesandmakeitthetargetdatacenter.Todoso,usearangosyncconfiguresync...asdescribedinReversingsynchronizationdirection.

Whattodoincaseofaplannednetworkoutage

AllArangoSynctaskssendoutheartbeatmessagesouttotheotherdatacentertoindicate"itisstillalive".Theotherdatacenterassumestheconnectionis"outofsync"whenitdoesnotreceiveanymessagesforacertainperiodoftime.

Ifyou'replanningsomesortofmaintenancewhereyouknowtheconnectivitywillbelostforsometime(e.g.3hours),youcanprepareArangoSyncforthatsuchthatitwillholdofre-synchronizationforagivenperiodoftime.

Todoso,onbothdatacenters,run:

arangosyncsetmessagetimeout\

--master.endpoint=<endpointsofsyncmastersinthedatacenter>\

ThelastargumentistheperiodthatArangoSyncshouldhold-ofresynchronizationfor.Thiscanbeminutes(e.g.15m)orhours(e.g.3h).

Ifmaintenanceistakinglongerthanexpected,youcanusethesamecommandtheextendtheholdofperiod(e.g.to4h).

Afterthemaintenance,usethesamecommandrestoretheholdofperiodtoitsdefaultof1h.

Whattodoincaseofadocumentthatexceedsthemessagequeuelimits

Ifyouinsert/updateadocumentinacollectionandthesizeofthatdocumentislargerthanthemaximummessagesizeofyourmessagequeue,thecollectionwillnolongerbeabletosynchronize.Itwillgointoafailedstate.

Torecoverfromthat,firstremovethedocumentfromtheArangoDBclusterinthesourcedatacenter.Afterthat,foreachfailedshard,run:

arangosyncresetfailedshard\

--master.endpoint=<endpointsofsyncmastersinthedatacenter>\

--database=<nameofthedatabase>\

--collection=<nameofthecollection>\

--shard=<indexoftheshard(startingat0)>

Afterthiscommand,anewsetoftaskswillbestartedtosynchronizetheshard.Itcantakesometimefortheshardtoreachrunningstate.

Monitoring

MonitoringdatacentertodatacenterreplicationThissectionincludesinformationrelatedtothemonitoringofthedatacentertodatacenterreplication.

MetricsArangoSync(master&worker)providemetricsthatcanbeusedformonitoringthedatacentertodatacenterrepliationsolution.ThesemetricsareavailableusingthefollowingHTTPSendpoints:

GET/metrics:ProvidesmetricsinaformatsupportedbyPrometheus.GET/metrics.json:ProvidesthesamemetricsinJSONformat.

Bothendpointsincludehelpinformationpermetrics.

Note:Bothendpointsrequireauthentication.Besidestheusualauthenticationmethodstheseendpointsarealsoaccessibleusingaspecialbearertokenspecifiedusingthe--monitoring.tokencommandlineoption.

ThePrometheusoutput(/metrics)lookslikethis:

#HELParangosync_master_worker_registrationsTotalnumberofregistrations

#TYPEarangosync_master_worker_registrationscounter

arangosync_master_worker_registrations2

#HELParangosync_master_worker_storageNumberoftimesworkerinfoisstored,loaded

#TYPEarangosync_master_worker_storagecounter

arangosync_master_worker_storage{kind="",op="save",result="success"}20

arangosync_master_worker_storage{kind="empty",op="load",result="success"}1

TheJSONoutput(/metrics.json)lookslikethis:

"arangosync_master_worker_registrations":{

"help":"Totalnumberofregistrations",

"type":"counter",

"samples":[

"value":2

"arangosync_master_worker_storage":{

"help":"Numberoftimesworkerinfoisstored,loaded",

"type":"counter",

"samples":[

"value":8,

"labels":{

"kind":"",

"op":"save",

"result":"success"

"value":1,

"labels":{

"kind":"empty",

"op":"load",

"result":"success"

Hint:Togetalistofametricsandtheirhelpinformation,run:

aliasjq='dockerrun--rm-irealguess/jqjq'

curl-sk-u"<user>:<password>"https://<syncmaster-IP>:8629/metrics.json|\

jq'with_entries({key:.key,value:.value.help})'

Security

DatacentertodatacenterSecurityThissectionincludesinformationrelatedtothedatacentertodatacenterreplicationsecurity.

Firewallsettings

ThecomponentsofArangoSyncuse(TCP)networkconnectionstocommunicatewitheachother.Belowyou'llfindanoverviewoftheseconnectionsandtheTCPportsthatshouldbeaccessible.

1. Thesyncmastersmustbeallowedtoconnecttothefollowingcomponentswithinthesamedatacenter:

ArangoDBagentsandcoordinators(defaultports:8531and8529)Kafkabrokers(defaultport9092)Syncworkers(defaultport8729)

Additionallythesyncmastersmustbeallowedtoconnecttothesyncmastersintheotherdatacenter.

Bydefaultthesyncmasterswilloperateonport8629.

2. Thesyncworkersmustbeallowedtoconnecttothefollowingcomponentswithinthesamedatacenter:

ArangoDBcoordinators(defaultport8529)Kafkabrokers(defaultport9092)Syncmasters(defaultport8629)

Bydefaultthesyncworkerswilloperateonport8729.

AdditionallythesyncworkersmustbeallowedtoconnecttotheKafkabrokersintheotherdatacenter.

3. Kafka

Thekafkabrokersmustbeallowedtoconnecttothefollowingcomponentswithinthesamedatacenter:

Otherkafkabrokers(defaultport9092)Zookeeper(defaultports2181,2888and3888)

Thedefaultportforkafkais9092.Thedefaultkafkainstallationwillalsoexposesomeprometheusmetricsonport7071.Togainmoreinsightintokafkaopenthisportforyourprometheusinstallation.

4. Zookeeper

Thezookeeperagentsmustbeallowedtoconnecttothefollowingcomponentswithinthesamedatacenter:

OtherzookeeperagentsThesetuphereisabitspecialaszookeeperuses3portsfordifferentoperations.Allagentsneedtobeabletoconnecttoalloftheseports.

BydefaultZookeeperuses:

port2181forclientcommunicationport2888forfollowercommunicationport3888forleaderelections

Certificates

DigitalcertificatesareusedinmanyplacesinArangoSyncforbothencryptionandauthentication.

InArangoSyncallnetworkconnectionsareusingTransportLayerSecurity(TLS),asetofprotocolsthatensurethatallnetworktrafficisencrypted.ForthisTLScertificatesareused.TheserversideofthenetworkconnectionoffersaTLScertificate.Thiscertificateis(often)verifiedbytheclientsideofthenetworkconnection,toensurethatthecertificateissignedbyatrustedCertificateAuthority(CA).Thisensurestheintegrityoftheserver.

Inseveralplacesadditionalcertificatesareusedforauthentication.Inthosecasestheclientsideoftheconnectionoffersaclientcertificate

(ontopofanexistingTLSconnection).Theserversideoftheconnectionusestheclientcertificatetoauthenticatetheclientand(optionally)decideswhichrightsshouldbeassignedtotheclient.

Note:ArangoSyncdoesallowtheuseofcertificatessignedbyawellknowCA(eg.verisign)howeveritismoreconvenient(andcommon)touseyourownCA.

Formats

Allcertificatesarex509certificateswithapublickey,aprivatekeyandanoptionalchainofcertificatesusedtosignthecertificate(thischainistypicallyprovidedbytheCertificateAuthority(CA)).Dependingontheiruse,certificatesstoredinadifferentformat.

Thefollowingformatsareused:

Publickeyonly(.crt):Afilethatcontainsonlythepublickeyofacertificatewithanoptionalchainofparentcertificates(publickeysofcertificatesusedtosignedthecertificate).Sincethisformatcontainsonlypublickeys,itisnotaproblemifitscontentsareexposed.Itmuststillbestoreitinasafeplacetoavoidlosingit.Privatekeyonly(.key):Afilethatcontainsonlytheprivatekeyofacertificate.Itisvitaltoprotectthesefilesandstoretheminasafeplace.Keyfilewithpublic&privatekey(.keyfile):Afilethatcontainsthepublickeyofacertificate,anoptionalchainofparentcertificatesandaprivatekey.Sincethisformatalsocontainsaprivatekey,itisvitaltoprotectthesefilesandstoretheminasafeplace.Javakeystore(.jks):Afilecontainingasetofpublicandprivatekeys.Itispossibletoprotectaccesstothecontentofthisfileusingakeystorepassword.Sincethisformatcancontainprivatekeys,itisvitaltoprotectthesefilesandstoretheminasafeplace(evenwhenitscontentisprotectedwithakeystorepassword).

Creatingcertificates

ArangoSyncprovidescommandstocreateallcertificatesneeded.

TLSservercertificates

TocreateacertificateusedforTLSserversinthekeyfileformat,youneedthepublickeyoftheCA(--cacert),theprivatekeyoftheCA(--cakey)andoneormorehostnames(orIPaddresses).Thenrun:

arangosynccreatetlskeyfile\

--cacert=my-tls-ca.crt--cakey=my-tls-ca.key\

--host=<hostname>\

--keyfile=my-tls-cert.keyfile

Makesuretostorethegeneratedkeyfile(my-tls-cert.keyfile)inasafeplace.

TocreateacertificateusedforTLSserversinthecrt&keyformat,youneedthepublickeyoftheCA(--cacert),theprivatekeyoftheCA(--cakey)andoneormorehostnames(orIPaddresses).Thenrun:

arangosynccreatetlscertificate\

--cacert=my-tls-ca.crt--cakey=my-tls-ca.key\

--host=<hostname>\

--cert=my-tls-cert.crt\

--key=my-tls-cert.key\

Makesuretoprotectandstorethegeneratedfiles(my-tls-cert.crt&my-tls-cert.key)inasafeplace.

Clientauthenticationcertificates

Tocreateacertificateusedforclientauthenticationinthekeyfileformat,youneedthepublickeyoftheCA(--cacert),theprivatekeyoftheCA(--cakey)andoneormorehostnames(orIPaddresses)oremailaddresses.Thenrun:

arangosynccreateclient-authkeyfile\

--cacert=my-client-auth-ca.crt--cakey=my-client-auth-ca.key\

[--host=<hostname>|--email=<emailaddress>]\

--keyfile=my-client-auth-cert.keyfile

Makesuretoprotectandstorethegeneratedkeyfile(my-client-auth-cert.keyfile)inasafeplace.

CAcertificates

TocreateaCAcertificateusedtosignTLScertificates,run:

arangosynccreatetlsca\

--cert=my-tls-ca.crt--key=my-tls-ca.key

Makesuretoprotectandstorebothgeneratedfiles(my-tls-ca.crt&my-tls-ca.key)inasafeplace.Note:CAcertificateshaveamuchlongerlifetimethannormalcertificates.Thereforeevenmorecareisneededtostorethemsafely.

TocreateaCAcertificateusedtosignclientauthenticationcertificates,run:

arangosynccreateclient-authca\

--cert=my-client-auth-ca.crt--key=my-client-auth-ca.key

Makesuretoprotectandstorebothgeneratedfiles(my-client-auth-ca.crt&my-client-auth-ca.key)inasafeplace.Note:CAcertificateshaveamuchlongerlifetimethannormalcertificates.Thereforeevenmorecareisneededtostorethemsafely.

Renewingcertificates

Allcertificateshavemetainformationinthemthelimittheiruseinfunction,target&lifetime.Acertificatecreatedforclientauthentication(function)cannotbeusedasaTLSservercertificate(sameistrueforthereverse).Acertificateforhostmyserver(target)cannotbeusedforhostanotherserver.AcertficiatethatisvaliduntilOctober2017(limetime)cannotbeusedafterOctober2017.

Ifanythingchangesinfunction,targetorlifetimeyouneedanewcertificate.

Theprocedureforcreatingarenewedcertificateisthesameasforcreatinga"first"certificate.Aftercreatingtherenewedcertificatetheprocess(es)usingthemhavetobeupdated.Thismeanrestartingthem.AllArangoSynccomponentsaredesignedtosupportstoppingandstartingsingleinstances,butdonotrestartmorethan1instanceatthesametime.Assoonas1instancehasbeenrestarted,giveitsometimeto"catchup"beforerestartingthenextinstance.

Architecture

AppendOnly/MVCC

Insteadofoverwritingexistingdocuments,ArangoDBwillcreateanewversionofmodifieddocuments.Thisiseventhecasewhenadocumentgetsdeleted.Thetwobenefitsare:

Objectscanbestoredcoherentlyandcompactlyinthemainmemory.Objectsarepreserved,isolatedwritingandreadingtransactionsallowaccessingtheseobjectsforparalleloperations.

Thesystemcollectsobsoleteversionsasgarbage,recognizingthemasforsaken.Garbagecollectionisasynchronousandrunsparalleltootherprocesses.

MostlyMemory/Durability

Databasedocumentsarestoredinmemory-mappedfiles.Perdefault,thesememory-mappedfilesaresyncedregularlybutnotinstantly.Thisisoftenagoodtradeoffbetweenstorageperformanceanddurability.Ifthislevelofdurabilityistoolowforanapplication,theservercanalsosyncallmodificationstodiskinstantly.ThiswillgivefulldurabilitybutwillcomewithaperformancepenaltyaseachdatamodificationwilltriggerasyncI/Ooperation.

Architecture

Write-aheadlogTheWrite-aheadlogispartoftheMMFilesstorageengine;Thisdoesn'tapplytoyourArangoDBifyouarerunningwiththeRocksDBstorageengine.

Startingwithversion2.2ArangoDBstoresalldata-modificationoperationinitswrite-aheadlog.Thewrite-aheadlogissequenceofappend-onlyfilescontainingallthewriteoperationsthatwereexecutedontheserver.

Itisusedtorundatarecoveryafteraservercrash,andcanalsobeusedinareplicationsetupwhenslavesneedtoreplaythesamesequenceofoperationsasonthemaster.

Bydefault,eachwrite-aheadlogfileis32MiBinsize.Thissizeisconfigurableviatheoption--wal.logfile-size.

Whenawrite-aheadlogfileisfull,itissettoread-only,andfollowingoperationswillbewrittenintothenextwrite-aheadlogfile.Bydefault,ArangoDBwillreservesomesparelogfilesinthebackgroundsoswitchinglogfilesshouldbefast.HowmanyreservelogfilesArangoDBwilltrytokeepavailableinthebackgroundcanbecontrolledbytheconfigurationoption--wal.reserve-logfiles.

Datacontainedinfullwrite-aheadfileswilleventuallybetransferredintothejournalsordatafilesofcollections.Onlythe"surviving"documentswillbecopiedover.Whenallremainingoperationsfromawrite-aheadlogfilehavebeencopiedoverintothejournalsordatafilesofthecollections,thewrite-aheadlogfilecansafelyberemovedifitisnotusedforreplication.

Long-runningtransactionspreventwrite-aheadlogfilesfrombeingfullygarbage-collectedbecauseitisunclearwhetheratransactionwillcommitorabort.Long-runningtransactionscanthusblockthegarbage-collectionprogressandshouldthereforebeavoidedatallcosts.

Onasystemthatactsasareplicationmaster,itisusefultokeepafewofthealreadycollectedwrite-aheadlogfilessoreplicationslavesstillcanfetchdatafromthemifrequired.Howmanycollectedlogfileswillbekeptbeforetheygetdeletedisconfigurableviatheoption--wal.historic-logfiles.

Forallwrite-aheadlogconfigurationoptions,pleaserefertothepageWrite-aheadlogoptions.

Write-aheadlog

StorageEnginesAttheverybottomoftheArangoDBdatabaseliesthestorageengine.Thestorageengineisresponsibleforpersistingthedocumentsondisk,holdingcopiesinmemory,providingindexesandcachestospeedupqueries.

Uptoversion3.1ArangoDBonlysupportedmemorymappedfiles(MMFiles)assolestorageengine.Beginningwith3.2ArangoDBhassupportforpluggablestorageengines.ThesecondsupportedengineisRocksDBfromFacebook.

MMFiles RocksDB

default optional

datasetneedstofitintomemory workwithasmuchdataasfitsondisk

indexesinmemory hotsetinmemory,dataandindexesondisk

slowrestartduetoindexrebuilding faststartup(norebuildingofindexes)

volatilecollections(onlyinmemory,optional) collectiondataalwayspersisted

collectionlevellocking(writesblockreads) concurrentreadsandwrites

Blogarticle:ComparingnewRocksDBandMMFilesstorageengines

RocksDBisanembeddablepersistentkey-valuestore.Itisalogstructuredatabaseandisoptimizedforfaststorage.

TheMMFilesengineisoptimizedfortheuse-casewherethedatafitsintothemainmemory.Itallowsforveryfastconcurrentreads.However,writesblockreadsandlockingisoncollectionlevel.Indexesarealwaysinmemoryandarerebuiltonstartup.Thisgivesbetterperformancebutimposesalongerstartuptime.

TheRocksDBengineisoptimizedforlargedata-setsandallowsforasteadyinsertperformanceevenifthedata-setismuchlargerthanthemainmemory.Indexesarealwaysstoredondiskbutcachesareusedtospeedupperformance.RocksDBusesdocument-levellocksallowingforconcurrentwrites.Writesdonotblockreads.Readsdonotblockwrites.

Theenginemustbeselectedforthewholeserver/cluster.Itisnotpossibletomixengines.Thetransactionhandlingandwrite-ahead-logformatintheindividualenginesisverydifferentandthereforecannotbemixed.

RocksDB

Advantages

RocksDBisaveryflexibleenginethatcanbeconfiguredforvarioususecases.

ThemainadvantagesofRocksDBare

document-levellockssupportforlargedata-setspersistentindexes

Caveats

RocksDBallowsconcurrentwrites.However,whentouchingthesamedocumentawriteconflictisraised.ThiscannothappenwiththeMMFilesengine,thereforeapplicationsthatswitchtoRocksDBneedtobepreparedthatsuchexceptioncanarise.ItispossibletoexclusivelylockcollectionswhenexecutingAQL.Thiswillavoidwriteconflictsbutalsoinhibitsconcurrentwrites.

Currently,anotherrestrictionisduetothetransactionhandlinginRocksDB.Transactionsarelimitedintotalsize.Ifyouhaveastatementmodifyingalotofdocumentsitisnecessarytocommitdatainbetween.ThiswillbedoneautomaticallyforAQLbydefault.

Performance

RocksDBisabasedonlog-structuredmergetree.Agoodintroductioncanbefoundin:

StorageEngines

http://www.benstopford.com/2015/02/14/log-structured-merge-trees/https://blog.acolyer.org/2014/11/26/the-log-structured-merge-tree-lsm-tree/

Thebasicideaisthatdataisorganizedinlevelswereeachlevelisafactorlargerthantheprevious.Newdatawillresideinsmallerlevelswhileolddataismoveddowntothelargerlevels.Thisallowstosupporthighrateofinsertsoveranextendedperiod.Inprincipleitispossiblethatthedifferentlevelsresideondifferentstoragemedia.ThesmalleronesonfastSSD,thelargeronesonbiggerspinningdisks.

RocksDBitselfprovidesalotofdifferentknobstofinetunethestorageengineaccordingtoyouruse-case.ArangoDBsupportsthemostcommononesusingtheoptionsbelow.

Performancereportsforthestorageenginecanbefoundhere:

https://github.com/facebook/rocksdb/wiki/performance-benchmarkshttps://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide

ArangoDBoptions

ArangoDBhasacacheforthepersistentindexesinRocksDB.Thetotalsizeofthiscacheiscontrolledbytheoption

--cache.size

RocksDBalsohasacachefortheblocksstoredondisk.Thesizeofthiscacheiscontrolledbytheoption

--rocksdb.block-cache-size

ArangoDBdistributestheavailablememoryequallybetweenthetwocachesbydefault.

ArangoDBchoosesasizeforthevariouslevelsinRocksDBthatissuitableforgeneralpurposeapplications.

RocksDBlogstrutureddatalevelshaveincreasingsize

MEM:--

L1:----

L2:--------

NeworupdatedDocumentsarefirststoredinmemory.Ifthismemtablereachesthelimitgivenby

--rocksdb.write-buffer-size

itwillconvertedtoanSSTfileandinsertedatlevel0.

Thefollowingoptioncontrolsthesizeofeachlevelandthedepth.

--rocksdb.num-levelsN

LimitsthenumberoflevelstoN.Bydefaultitis7andthereisseldomareasontochangethis.Anewlevelisonlyopenedifthereistoomuchdatainthepreviousone.

--rocksdb.max-bytes-for-level-baseB

L0willholdatmostBbytes.

--rocksdb.max-bytes-for-level-multiplierM

EachlevelisatmostMtimesasmuchbytesasthepreviousone.ThereforethemaximumnumberofbytesforlevelLcanbecalculatedas

max-bytes-for-level-base*(max-bytes-for-level-multiplier^(L-1))

StorageEngines

Future

RocksDBimposesalimitonthetransactionsize.Itisoptimizedtohandlesmalltransactionsveryefficiently,butiseffectivelylimitingthetotalsizeoftransactions.

ArangoDBcurrentlyusesRocksDB'stransactionstoimplementtheArangoDBtransactionhandling.ThereforethesamerestrictionsapplyforArangoDBtransactionswhenusingtheRocksDBengine.

WewillimprovethisbyintroducingdistributedtransactionsinafutureversionofArangoDB.ThiswillallowhandlinglargetransactionsasaseriesofsmallRocksDBtransactionsandhenceremovingthesizerestriction.

StorageEngines

ReleaseNotes

WhatsNew

WhatsNewin3.3WhatsNewin3.2WhatsNewin3.1WhatsNewin3.0WhatsNewin2.8WhatsNewin2.7WhatsNewin2.6WhatsNewin2.5WhatsNewin2.4WhatsNewin2.3WhatsNewin2.2WhatsNewin2.1

KnownIssues

KnownIssuesin3.2

Incompatiblechanges

AlsoseeUpgradingintheAdministrationchapter.

Incompatiblechangesin3.3Incompatiblechangesin3.2Incompatiblechangesin3.1Incompatiblechangesin3.0Incompatiblechangesin2.8Incompatiblechangesin2.7Incompatiblechangesin2.6Incompatiblechangesin2.5Incompatiblechangesin2.4Incompatiblechangesin2.3

Releasenotes

FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB3.3.ArangoDB3.3alsocontainsseveralbugfixesthatarenotlistedhere.

Datacenter-to-datacenterreplication(DC2DC)

Everycompanyneedsadisasterrecoveryplanforallimportantsystems.Thisistruefromsmallunitslikesingleprocessesrunninginsomecontainertothelargestdistributedarchitectures.Fordatabasesinparticularthisusuallyinvolvesamixtureoffault-tolerance,redundancy,regularbackupsandemergencyplans.Thelargeradatastore,themoredifficultitistocomeupwithagoodstrategy.

Therefore,itisdesirabletobeabletorunadistributeddatabaseinonedata-centerandreplicatealltransactionstoanotherdata-centerinsomeway.Often,transactionlogsareshippedoverthenetworktoreplicateeverythinginanother,identicalsystemintheotherdata-center.Somedistributeddatastoreshavebuilt-insupportformultipledata-centerawarenessandcanreplicatebetweendata-centersinafullyautomaticfashion.

ArangoDB3.3takesanevolutionarystepforwardbyintroducingmulti-data-centersupport,whichisasynchronousdata-centertodata-centerreplication.Oursolutionisasynchronousandscalestoarbitraryclustersizes,providedyournetworklinkbetweenthedata-centershasenoughbandwidth.Itisfault-tolerantwithoutasinglepointoffailureandincludesalotofmetricsformonitoringinaproductionscenario.

DC2DCisavailableintheEnterpriseedition.

Encryptedbackups

ArangodumpcannowcreateencryptedbackupsusingAES256forencryption.Theencryptionkeycanbereadfromafileorfromageneratorprogram.Itworksinsingleserverandclustermode.

Examplefornon-encryptedbackup(everyonewithaccesstothebackupwillbeabletoreadit):

arangodump--collection"secret"dump

Inordertocreateanencryptedbackup,addthe--encryption.keyfileoptionwheninvokingarangodump:

arangodump--collection"secret"dump--encryption.keyfile~/SECRET-KEY

Thekeymustbeexactly32byteslong(requiredbytheAESblockcipher).

Notethatarangodumpwillnotstorethekeyanywhere.Itistheresponsibilityoftheusertofindasafeplaceforthekey.However,arangodumpwillstoretheusedencryptionmethodinafilenamedENCRYPTIONinthedumpdirectory.Thatwayarangorestorecanlaterfindoutwhetheritisdealingwithanencrypteddumpornot.

Tryingtorestoretheencrypteddumpwithoutspecifyingthekeywillfail:

arangorestore--collection"secret-collection"dump--create-collectiontrue

arangorestorewillcomplainwith:

thedumpdataseemstobeencryptedwithaes-256-ctr,butnokeyinformationwasspecifiedtodecryptthedumpitisrecommendedtospecifyeither--encryption.key-fileor--encryption.key-generatorwheninvokingarangorestorewithanencrypteddump

Itisrequiredtousetheexactsamekeywhenrestoringthedata.Againthisisdonebyprovidingthe--encryption.keyfileparameter:

arangorestore--collection"secret-collection"dump--create-collectiontrue--encryption.keyfile~/SECRET-KEY

WhatsNewin3.3

Usingadifferentkeywillleadtothebackupbeingnon-recoverable.

NotethatencryptedbackupscanbeusedtogetherwiththealreadyexistingRocksDBencryption-at-restfeature,buttheycanalsobeusedfortheMMFilesengine,whichdoesnothaveencryption-at-rest.

EncryptedbackupsareavailableintheEnterpriseedition.

Server-levelreplicationArangoDBsupportsasynchronousreplicationfunctionalitysinceversion1.4,butreplicatingfromamasterserverwithmultipledatabasesrequiredmanualsetupontheslaveforeachindividualdatabasetoreplicate.Whenanewdatabasewascreatedonthemaster,oneneededtotakeactionontheslavetoensurethatdataforthatdatabasegotactuallyreplicated.Replicationontheslavealsowasnotawareofwhenadatabasewasdroppedonthemaster.

3.3addsserver-levelreplication,whichwillreplicatethecurrentandfuturedatabasesfromthemastertotheslaveautomaticallyaftertheinitialsetup.

Inordertosetupglobalreplicationona3.3slaveforalldatabasesofagiven3.3master,thereisnowtheso-calledglobalApplier.Ithasthesameinterfaceastheexistingapplier,butitwillreplicatefromalldatabasesofthemasterandnotjustasingleone.

Inordertostartthereplicationontheslaveandmakeitreplicatealldatabasesfromagivenmaster,usethesecommandsontheslave:

replication.setupReplicationGlobal({

endpoint:"tcp://127.0.0.1:8529",

username:"root",

password:"",

autoStart:true

Tocheckiftheapplierisrunning,alsousetheglobalApplierobject:

replication.globalApplier.state().state

Theserver-levelreplicationrequiresboththemasterandslaveserverstoArangoDBversion3.3orhigher.

Asynchronousfailover

Aresilientsetupcannoweasilybeachievedbyrunningapairofconnectedservers,ofwhichoneinstancebecomesthemasterandtheotheranasynchronouslyreplicatingslave,withautomaticfailoverbetweenthem.

Twoserversareconnectedviaasynchronousreplication.Oneoftheserversiselectedleader,andtheotheroneismadeafollowerautomatically.Atstartup,thetwoserversfightforleadership.Thefollowerwillautomaticallystartreplicationfromthemasterforallavailabledatabases,usingtheserver-levelreplicationintroducedin3.3.

Whenthemastergoesdown,thisisautomaticallydetectedbyanagencyinstance,whichisalsostartedinthismode.Thisinstancewillmakethepreviousfollowerstopitsreplicationandmakeitthenewleader.

Thefollowerwillautomaticallydenyallreadandwriterequestsfromclientapplications.Onlythereplicationitselfisallowedtoaccessthefollower'sdatauntilthefollowerbecomesanewleader.

Whensendingarequesttoreadorwritedataonafollower,thefollowerwillalwaysrespondwithHTTP503(Serviceunavailable)andprovidetheaddressofthecurrentleader.Clientapplicationsanddriverscanusethisinformationtothenmakeafollow-uprequesttotheproperleader:

HTTP/1.1503ServiceUnavailable

X-Arango-Endpoint:http://[::1]:8531

WhatsNewin3.3

Clientapplicationscanalsodetectwhothecurrentleaderandthefollowersarebycallingthe/_api/cluster/endpointsRESTAPI.ThisAPIisaccessibleonleadersandfollowersalike.

TheArangoDBstartersupportsstartingtwoserverswithasynchronousreplicationandfailoveroutofthebox.

ThearangojsdriverforJavaScript,theGodriverandtheJavadriverforArangoDBsupportautomaticfailoverincasethecurrentlyaccessedserverendpointrespondswithHTTP503.

Blogarticle:IntroducingthenewArangoDBJavadriverwithloadbalancingandadvancedfallback

RocksDBthrottling

ArangoDB3.3allowswriteoperationstotheRocksDBenginebethrottled,inordertopreventlongerwritestalls.Thethrottlingisadaptive,meaningthatitautomaticallyadaptstotheactualwriterate.Thisresultsinmuchmorestableresponsetimes,whichisbetterforclientapplicationsandclusterhealthtests,becausetimeoutscausedbywritestallsarelesslikelytooccurandtheserverthusnotmistakenlyassumedtobedown.

Blogarticle:RocksDBsmoothingforArangoDBcustomers

FastershardcreationinclusterWhenusingacluster,onenormallywantsresilience,soreplicationFactorissettoatleast2.Thenumberofshardsisoftensettoratherhighvalueswhencreatingcollections.

Creatingacollectionintheclusterwillmakethecoordinatorstorethesetupmetadataofthenewcollectionintheagencyfirst.Subsequentiallyalldatabaseserversoftheclusterwilldetectthatthereisworktodoandwillbegincreatingtheshards.Thiswillfirsthappenfortheshardleaders.Foreachshardleaderthatfinisheswiththesetup,thesynchronousreplicationwithitsfollowersisthenestablished.Thatwillmakesurethateveryfuturedatamodificationwillnotbecomeeffectiveontheleaderonly,butalsoonallthefollowers.

In3.3thissetupprotocolhasgotsomeshortcutsfortheinitialshardcreation,whichspeedsupcollectioncreationbyroughly50to60percent.

LDAPauthentication

TheLDAPauthenticationmoduleintheEnterpriseeditionhasbeenenhanced.Thefollowingoptionshavebeenaddedtoit:

theoption--server.local-authenticationcontrolswhetherthelocal_userscollectionisalsousedforlookingupusers.Thisisalsothedefaultbehavior.IftheauthenticationshallberestrictedtojusttheLDAPdirectory,theoptioncanbesettotrue,andarangodwillthennotmakeanyqueriestoits_userscollectionwhenlookingupusers.

theoption--server.authentication-timeoutcontrolstheexpirationtimeforcachedLDAPuserinformationentriesinarangod.

basicrolesupporthasbeenaddedfortheLDAPmoduleintheEnterpriseedition.NewconfigurationoptionsforLDAPin3.3are:

--ldap.roles-attribute-name

--ldap.roles-transformation

--ldap.roles-search

--ldap.roles-include

--ldap.roles-exclude

--ldap.superuser-role

PleaserefertoLDAPforadetailedexplanation.

Miscellaneousfeatures

whencreatingacollectioninthecluster,thereisnowanoptionalparameterenforceReplicationFactor:whenset,thisparameterenforcesthatthecollectionwillonlybecreatediftherearenotenoughdatabaseserversavailableforthedesiredreplicationFactor.

AQLDISTINCTisnotchangingtheorderofprevious(sorted)results

WhatsNewin3.3

PreviouslytheimplementationofAQLdistinctstoredallencounteredvaluesinahashtableinternally.Whendone,thefinalresultswerereturnedintheorderdictatedbythehashtablethatwasusedtostorethekeys.Thisorderwasmoreorlessunpredictable.Thoughthiswasdocumentedbehavior,itwasinconvenientforendusers.

3.3nowdoesnotchangethesortorderoftheresultanymorewhenDISTINCTisused.

SeveralAQLfunctionshavebeenimplementedinC++,whichcanhelpsavememoryandCPUtimeforconvertingthefunctionargumentsandresults.Thefollowingfunctionshavebeenported:

LEFTRIGHTSUBSTRINGTRIMMATCHES

TheArangoShellpromptsubstitutioncharactershavebeenextended.Nowthefollowingextrasubstitutionscanbeusedforthearangoshprompt:

'%t':currenttimeastimestamp'%a':elpasedtimesinceArangoShellstartinseconds'%p':durationoflastcommandinseconds

Forexample,toshowtheexecutiontimeofthelastcommandexecutedinarangoshintheshell'sprompt,startarangoshusing:

arangosh--console.prompt"%E@%d%p>"

Therearenewstartupoptionsfortheloggingtoaiddebugginganderrorreporting:

--log.role:willshowone-lettercodeofserverrole(A=agent,C=coordinator,...)Thisisespeciallyusefulwhenaggregatinglogs.

Theexistingrolesusedinlogsare:

U:undefined/unclear(usedatstartup)S:singleserverC:coordinatorP:primaryA:agent

--log.line-numbertrue:thisoptionwillnowadditionallyshowthenameoftheC++functionthattriggeredthelogmessage(filenameandlinenumberwerealreadyloggedinpreviousversions)

--log.thread-nametrue:thisnewoptionwilllogthenameoftheArangoDBthreadthattriggeredthelogmessage.WillhavemeaningfuloutputonLinuxonly

maketheArangoShell(arangosh)refillitscollectioncachewhenayet-unknowncollectionisfirstaccessed.Thisfixesthefollowingproblemwhenworkingwiththeshellwhileinanothershellorbyanotherprocessanewcollectionisadded:

arangosh1>db._collections();//shell1listsallcollections

arangosh2>db._create("test");//shell2nowcreatesanewcollection'test'

arangosh1>db.test.insert({});//shell1isnotawareofthecollectioncreated

//inshell2,sotheinsertwillfail

WhatsNewin3.3

IncompatiblechangesinArangoDB3.3ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB3.3,andadjustanyclientprogramsifnecessary.

ThefollowingincompatiblechangeshavebeenmadeinArangoDB3.3:

AQL:duringatraversalifavertexisnotfound,arangodwillnotloganerrorandcontinuewithaNULLvalue,butwillinsteadregisterawarningatthequeryandcontinuewithaNULLvalue.

Ifanon-existingvertexisreferencedfromatraversal,itisnotdesirabletologerrorsasArangoDBcanstoreedgespointingtonon-existingvertices(whichisperfectlyvalidifthelow-levelinsertAPIsareused).Aslinkingtonon-existingverticesmayindicateanissuein/withthedatamodelortheclientapplication,thewarningisregisteredinthequerysoclientapplicationshaveaccesstoit.

ArangoDBusernamesmustnotstartwiththestring:role:.

Thestartupconfigurationparameter--cluster.my-iddoesnothaveanyeffectin3.3.Forcompatibilityreasons,ArangoDB3.3willnotfailonstartupiftheoptionisstillusedintheconfiguration,butitwillsilentlyignorethisoption.

Thestartupconfigurationparameter--cluster.my-local-infoisdeprecatednow.Usingitwillmakearangodlogawarningonstartup.

Serverstartup:therecommendedvaluefortheLinuxkernelsettingin/proc/sys/vm/max_map_countwasincreasedtoavalueeighttimesashighasin3.2.arangodcomparesatstartupiftheeffectivevalueofthissettingispresumablytoolow,anditwillissueawarninginthiscase,recommendingtoincreasethevalue.

Thisisnowmorelikelytohappenthaninpreviousversions,astherecommendedvalueisnoweighttimeshigherthanin3.2.Thestartupwarningswilllooklikethis(withactualnumbersvarying):

WARNING{memory}maximumnumberofmemorymappingsperprocessis65530,whichseemstoolow.itisrecommendedtosetit

toatleast512000

PleaserefertotheLinuxkerneldocumentationformoreinformationonthissetting.ThischangeonlyaffectstheLinuxversionofArangoDB.

Clienttools

Theoption--recycle-idshasbeenremovedfromthearangorestorecommand.Usingthisoptioncouldhaveledtoproblemsontherestore,withpotentialidconflictsbetweentheoriginatingserver(thesourcedumpserver)andthetargetserver(therestoreserver).

Theoption--compathasbeenremovedfromthearangodumpcommandandthe/_api/replication/dumpRESTAPIendpoint.InordertocreateadumpfromanArangoDB2.8instance,pleaseuseanolderversionoftheclienttools.OlderArangoDBversionsarenolongerbesupportedbythearangodumpandarangorestorebinariesshippedwith3.3.

Miscellaneous

TheminimumsupportedcompilerforcompilingArangoDBfromsourceisnowg++5.4(bumpedupfromg++4.9).ThischangeonlyaffectsusersthatcompileArangoDBontheirown.

Storageengines

ArangoDB3.2offerstwostorageengines:

thealways-existingmemory-mappedfilesstorageengineanewstorageenginebasedonRocksDB

Memory-mappedfilesstorageengine(MMFiles)

Theformerstorageengine(namedMMFilesenginehenceforth)persistsdatainmemory-mappedfiles.

Anydatachangesaredonefirstintheengine'swrite-aheadlog(WAL).TheWALisreplayedafteracrashsotheengineoffersdurabilityandcrash-safety.DatafromtheWALiseventuallymovedtocollection-specificdatafiles.Thefilesarealwayswritteninanappend-onlyfashion,sodatainfilesisneveroverwritten.Obsoletedatainfileswilleventuallybepurgedbybackgroundcompactionthreads.

Mostofthisengine'sindexesarebuiltinRAM.Whenacollectionisloaded,thisrequiresrebuildingtheindexesinRAMfromthedatastoredondisk.TheMMFilesenginehascollection-levellocking.

Thisstorageengineisagoodchoicewhendata(includingtheindexes)canfitintheserver'savailableRAM.Ifthesizeofdataplusthein-memoryindexesexceedsthesizeoftheavailableRAM,thenthisenginemaytrytoallocatemorememorythanavailable.Thiswilleithermaketheoperatingsystemswapoutpartsofthedata(andcausediskI/O)or,whennoswapspaceisconfigured,invoketheoperatingsystem'sout-of-memoryprocesskiller.

Thelockingstrategyallowsparallelreadsandisoftengoodenoughinread-mostlyworkloads.Writesneedexclusivelocksonthecollections,sotheycanblockotheroperationsinthesamecollection.Thelockingstrategyalsoprovidestransactionalconsistencyandisolation.

RocksDBstorageengine

TheRocksDBstorageengineisnewinArangoDB3.2.Itisdesignedtostoredatasetsthatarebiggerthantheserver'savailableRAM.Itpersistsalldata(includingtheindexes)inaRocksDBinstance.

ThatmeansanydocumentreadorwriteoperationswillbeansweredbyRocksDBunderthehood.RocksDBwillservethedatafromitsownin-RAMcachesorfromdisk.TheRocksDBenginehasawrite-aheadlog(WAL)andusesbackgroundthreadsforcompaction.Itsupportsdatacompression.

TheRocksDBstorageenginehasdocument-levellocking.Readoperationsdonotblockandareneverblockedbyotheroperations.Writeoperationsonlyblockwritesonthesamedocuments/indexvalues.Becausemultiplewriterscanoperateinparallelonthesamecollection,thereisthepossibilityofwrite-writeconflicts.Ifsuchwriteconflictisdetected,oneofthewriteoperationsisabortedwitherror1200("conflict").Clientapplicationscantheneitheraborttheoperationorretry,basedontherequiredconsistencysemantics.

Storageengineselection

ThestorageenginetouseinanArangoDBclusterorasingle-serverinstancemustbeselectedinitially.ThedefaultstorageengineinArangoDB3.2istheMMFilesengineifnostorageengineisselectedexplicitly.Thisensuresallusersupgradingfromearlierversionscancontinuewiththewell-knownMMFilesengine.

Toselectthestorage-engine,thereistheconfigurationoption--server.storage-engine.Itcanbesettoeithermmfiles,rocksdborauto.Whilethefirsttwovalueswillexplicitlyselectastorageengine,theautooptionwillautomaticallychoosethestorageenginebasedonwhichstorageenginewaspreviouslyselected.Ifnoenginewasselectedpreviously,autowillselecttheMMFilesengine.Ifanenginewaspreviouslyselected,theselectionwillbewrittentoafileENGINEintheserver'sdatabasedirectoryandwillbereadfromthereatanysubsequentserverstarts.

WhatsNewin3.2

Oncethestorageenginewasselected,theselectioncannotbechangedbyadjusting--server.storage-engine.Inordertoswitchtoanotherstorageengine,itisrequiredtore-starttheserverwithanother(empty)databasedirectory.Inordertousedatacreatedwiththeotherstorageengine,itisrequiredtodumpthedatafirstwiththeoldengineandrestoreitusingthenewstorageengine.Thiscanbeachievedviainvokingarangodumpandarangorestore.

UnlikeinMySQL,thestorageengineselectioninArangoDBisforanentireclusteroranentiresingle-serverinstance.Alldatabasesandcollectionswillusethesamestorageengine.

RocksDBstorageengine:supportedindextypes

TheexistingindexesintheRocksDBengineareallpersistent.Thefollowingindexesaresupportedthere:

primary:thistypeofindexisautomaticallycreated.Itindexes_id/_key

edge:thisindexisautomaticallycreatedforedgecollections.Itindexes_fromand_to

hash,skiplist,persistent:theseareuser-definedindexes,Despitetheirnames,theyareneitherhashnorskiplistindexes.TheseindextypesmaptothesameRocksDB-basedsortedindeximplementation.Thesameistrueforthe"persistent"index.Thenames"hash","skiplist"and"persistent"areonlyusedforcompatibilitywiththeMMFilesenginewheretheseindexesexistedinpreviousandthecurrentversionofArangoDB.

geo:user-definedindexforproximitysearches

fulltext:user-definedsortedrevertedindexonwordsoccurringindocuments

WithSatelliteCollections,youcandefinecollectionstoshardtoaclusterandcollectionstoreplicatetoeachmachine.TheArangoDBqueryoptimizerknowswhereeachshardislocatedandsendstherequeststotheDBServersinvolved,whichthenexecutesthequery,locally.Withthisapproach,networkhopsduringjoinoperationsonshardedcollectionscanbeavoidedandresponsetimescanbeclosetothatofasingleinstance.

SatellitecollectionsareavailableintheEnterpriseedition.

MemorymanagementmakearangodstartwithlessV8JavaScriptcontexts

Thisspeedsuptheserverstartandmakesarangoduselessmemoryatstart.WheneveraV8contextisneededbyaFoxxactionorsomeotherJavaScriptoperationandthereisnousableV8context,anewcontextwillbecreateddynamicallynow.

Upto--javascript.v8-contextsV8contextswillbecreated,sothisoptionwillchangeitsmeaning.PreviouslyasmanyV8contextsasspecifiedbythisoptionwerecreatedatserverstart,andthenumberofV8contextsdidnotchangeatruntime.NowuptothisnumberofV8contextswillbeinuseatthesametime,buttheactualnumberofV8contextsisdynamic.

ThegarbagecollectorthreadwillautomaticallydeleteunusedV8contextsafterawhile.Thenumberofsparecontextswillgodowntoasfewasconfiguredinthenewoption--javascript.v8-contexts-minimum.ActuallythatmanyV8contextsarealsocreatedatserverstart.

ThefirstfewrequestsinnewV8contextsmaytakelongerthanincontextsthathavebeentherealready.PerformancemaythereforesufferabitfortheinitialrequestssenttoArangoDBorwhenthereareonlyfewbutperformance-criticalsituationsinwhichnewV8contextsneedtobecreated.Ifthisisaconcern,itcaneasilybefixedbysetting--javascipt.v8-contexts-minimumand--javascript.v8-contextstoarelativelyhighvalue,whichwillguaranteethatmanynumberofV8contextstobecreatedatstartupandkeptaroundevenwhenunused.

WaitingforanunusedV8contextwillnowalsoabortandwritealogmessageincasenoV8contextcanbeacquired/createdafter60seconds.

thenumberofpendingoperationsinarangodcannowbelimitedtoaconfigurablenumber.Ifthisnumberisexceeded,theserverwillnowrespondwithHTTP503(serviceunavailable).Themaximumsizeofpendingoperationsiscontrolledviathestartupoption--server.maximal-queue-size.Settingitto0means"nolimit".

WhatsNewin3.2

thein-memorydocumentrevisionscachewasremovedentirelybecauseitdidnotprovidetheexpectedbenefits.The3.1implementationshadoweddocumentdatainRAM,whichincreasedtheserver'sRAMusagebutdidnotspeedupdocumentlookupstoomuch.

Thisalsoobsoletesthestartupoptions--database.revision-cache-chunk-sizeand--database.revision-cache-target-size.

TheMMFilesenginenowdoesnotuseadocumentrevisionscachebuthasin-memoryindexesandmapsdocumentstoRAMautomaticallyviammapwhendocumentsareaccessed.TheRocksDBenginehasitsownmechanismforcachingaccesseddocuments.

CommunicationLayerHTTPresponsesreturnedbyarangodwillnowincludetheextraHTTPheaderx-content-type-options:nosnifftoworkaroundacross-sitescriptingbuginMSIE

thedefaultvaluefor--ssl.protocolwaschangedfromTLSv1toTLSv1.2.Whennotexplicitlyset,arangodandallclienttoolswillnowuseTLSv1.2.

theJSONdatainallincomingHTTPrequestsinnowvalidatedforduplicateattributenames.

IncomingJSONdatawithduplicateattributenameswillnowberejectedasinvalid.PreviousversionsofArangoDBonlyvalidatedtheuniquenessofattributenamesinsideincomingJSONforsomeAPIendpoints,butnotconsistentlyforallAPIs.

InternalJavaScriptRESTactionswillnowhidetheirstacktracestotheclientunlessinHTTPresponses.Insteadtheywillalwayslogtothelogfile.

JavaScript

updatedV8versionto5.7.0.0

changeundocumentedbehaviourincaseofinvalidrevisionidsinIf-MatchandIf-None-Matchheadersfrom400(BAD)to412(PRECONDITIONFAILED).

changedefaultstringtruncationlengthfrom80charactersto256charactersforprint/printShellfunctionsinArangoShellandarangod.Thiswillemitlongerprefixesofstringvaluesbeforetruncatingthemwith...,whichishelpfulfordebugging.ThischangeismostlyusefulwhenusingtheArangoShell(arangosh).

the@arangodbmodulenowprovidesatimefunctionwhichreturnsthecurrenttimeinsecondsasafloatingpointvaluewithmicrosecondprecision.

FoxxThereisnowanofficialHTTPAPIformanagingservices,allowingservicestobeinstalled,modified,uninstalledandreconfiguredwithouttheadministrativewebinterface.

ItisnowpossibletouploadasingleJavaScriptfileinsteadofaziparchiveifyourservicerequiresnoconfiguration,additionalfilesorsetup.Aminimalmanifestwillbegeneratedautomaticallyuponinstallationandtheuploadedfilewillbeusedastheservice'smainentrypoint.

DistributedGraphProcessing

WeaddedsupportforexecutingdistributedgraphalgorithmsakaPregel.Userscanrunarbitraryalgorithmsonanentiregraph,includinginclustermode.Weimplementedanumberofalgorithmsforvariouswell-knowngraphmeasures:

ConnectedComponentsPageRankShortestPathsCentralityMeasures(CentralityandBetweeness)

WhatsNewin3.2

CommunityDetection(viaLabelPropagation,Speakers-ListenersLabelPropagationorDMID)Userscancontributetheirownalgorithms

Optimizerimprovements

GeoindexesarenowimplicitlyandautomaticallyusedwhenusingappropriateSORT/FILTERstatementsinAQL,withouttheneedtousethesomewhatlimitedspecial-purposegeoAQLfunctionsNEARorWITHIN.

ComparedtousingthespecialpurposeAQLfunctionsthisapproachhastheadvantagethatitismorecomposable,andwillalsohonoranyLIMITvaluesusedintheAQLquery.

ThespecialpurposeNEARAQLfunctioncannowbesubstitutedwiththefollowingAQL(providedthereisageoindexpresentonthedoc.latitudeanddoc.longitudeattributes):

FORdocingeoSort

SORTDISTANCE(doc.latitude,doc.longitude,0,0)

LIMIT5

RETURNdoc

WITHINcanbesubstitutedwiththefollowingAQL:

FORdocingeoFilter

FILTERDISTANCE(doc.latitude,doc.longitude,0,0)<2000

RETURNdoc

Miscellaneousimprovements

addedREGEX_REPLACEAQLfunction

REGEX_REPLACE(text,search,replacement,caseInsensitive)→string

Replacethepatternsearchwiththestringreplacementinthestringtext,usingregularexpressionmatching.

text(string):thestringtosearchinsearch(string):aregularexpressionsearchpatternreplacement(string):thestringtoreplacethesearchpatternwithreturnsstring(string):thestringtextwiththesearchregexpatternreplacedwiththereplacementstringwhereverthepatternexistsintext

addednewstartupoption--query.fail-on-warningtomakeAQLqueriesabortinsteadofcontinuingwithwarnings.

Whensettotrue,thiswillmakeanAQLquerythrowanexceptionandabortincaseawarningoccurs.Thisoptionshouldbeusedindevelopmenttocatcherrorsearly.Ifsettofalse,warningswillnotbepropagatedtoexceptionsandwillbereturnedwiththequeryresults.Thestartupoptioncanalsobeoverridenonaperquery-level.

theslowquerylistnowcontainsthevaluesofbindvariablesusedintheslowqueries.Bindvariablesarealsoprovidedforthecurrentlyrunningqueries.Thishelpsdebuggingsloworblockingqueriesthatusedynamiccollectionnamesviabindparameters.

AQLbreakingchangeincluster:TheSHORTEST_PATHstatementusingedgecollectionnamesinsteadofagraphnamesnowrequirestoexplicitlynamethevertexcollectionnameswithintheAQLqueryinthecluster.ItcanbedonebyaddingWITH<name>atthebeginningofthequery.

Example:

FORv,eINOUTBOUNDSHORTEST_PATH@startTO@targetedges[...]

Nowhastobe:

WITHvertices

WhatsNewin3.2

Thischangeisduetoavoiddeadlocksitationsinclusteredcase.Anerrorstatingtheaboveisincluded.

Clienttools

addeddataexporttool,arangoexport.

arangoexportcanbeusedtoexportcollectionstojson,jsonlorxmlandexportagraphorcollectionstoxgmml.

added"jsonl"asinputfiletypeforarangoimp

added--translateoptionforarangoimptotranslateattributenamesfromtheinputfilestoattriubtenamesexpectedbyArangoDB

The--translateoptioncanbespecifiedmultipletimes(oncepertranslationtobeexecuted).Thefollowingexamplerenamesthe"id"columnfromtheinputfileto"_key",andthe"from"columnto"_from",andthe"to"columnto"_to":

arangoimp--typecsv--filedata.csv--translate"id=_key"--translate"from=_from"--translate"to=_to"

--translateworksforCSVandTSVinputsonly.

added--threadsoptiontoarangoimptospecifythenumberofparallelimportthreads

changeddefaultvalueforclienttoolsoption--server.max-packet-sizefrom128MBto256MB.thisallowstransferringbiggerresultsetsfromtheserverwithouttheclienttoolsrejectingthemasinvalid.

Authentication

addedLDAPauthentication(Enterpriseonly)

Authorization

addedreadonlymodeforuserscollectionlevelauthorizationrights

Readmoreintheoverview.

Foxxthecookiesessiontransportnowsupportsalloptionssupportedbythecookiemethodoftheresponseobject.

it'snowpossibletoprovideyourownversionofthegraphql-syncmodulewhenusingtheGraphQLextensionsforFoxxbypassingacopyofthemoduleusingthenewgraphqloption.

customAPIendpointscannowbetaggedusingthetagmethodtogenerateacleanerSwaggerdocumentation.

MiscellaneousChanges

arangodnowvalidatesseveralOS/environmentsettingsonstartupandwarnsifthesettingsarenon-ideal.Itadditionallywillprintoutwaystoremedytheoptions.

MostofthechecksareexecutedonLinuxsystemsonly.

added"deduplicate"attributeforarrayindexes,whichcontrolswhetherinsertingduplicateindexvaluesfromthesamedocumentintoauniquearrayindexwillleadtoanerrorornot:

//withdeduplicate=true,whichisthedefaultvalue:

db._create("test");

db.test.ensureIndex({type:"hash",fields:["tags[*]"],deduplicate:true});

db.test.insert({tags:["a","b"]});

WhatsNewin3.2

db.test.insert({tags:["c","d","c"]});//willwork,becausededuplicate=true

db.test.insert({tags:["a"]});//willfail

//withdeduplicate=false

db._create("test");

db.test.ensureIndex({type:"hash",fields:["tags[*]"],deduplicate:false});

db.test.insert({tags:["a","b"]});

db.test.insert({tags:["c","d","c"]});//willnotwork,becausededuplicate=false

db.test.insert({tags:["a"]});//willfail

WhatsNewin3.2

KnownIssuesThefollowingknownissuesarepresentinthisversionofArangoDBandwillbefixedinfollow-upreleases:

RocksDBstorageengine

TheRocksDBstorageengineisintentionallymissingthefollowingfeaturesthatarepresentintheMMFilesengine:

thedatafiledebugger(arango-dfdb)cannotbeusedwiththisstorageengine

RocksDBhasitsowncrashrecoverysousingthedfdbwillnotmakeanysensehere.

APIsthatreturncollectionpropertiesorfigureswillreturnslightlydifferentattributesfortheRocksDBenginethanfortheMMFilesengine.Forexample,theattributesjournalSize,doCompact,indexBucketsandisVolatilearepresentintheMMFilesenginebutnotintheRocksDBengine.ThememoryusagefiguresreportedforcollectionsintheRocksDBengineareestimatevalues,whereastheyareexactfortheMMFilesengine.

theRocksDBenginedoesnotsupportsomeoperationswhichonlymakesenseinthecontextoftheMMFilesengine.Theseare:

therotatemethodoncollectionstheflushmethodforWALfiles

theRocksDBstorageenginedoesnotsupportvolatilecollections

transactionsarelimitedinsize.Transactionsthatgettoobig(intermsofnumberofoperationsinvolvedorthetotalsizeofdatamodifiedbythetransaction)willbecommittedautomatically.EffectivelythismeansthatbigusertransactionsaresplitintomultiplesmallerRocksDBtransactionsthatarecommittedindividually.TheentireusertransactionwillnotnecessarilyhaveACIDpropertiesinthiscase.

Thethresholdvaluesfortransactionsizescanbeconfiguredgloballyusingthestartupoptions

--rocksdb.intermediate-commit-size:ifthesizeofalloperationsinatransactionreachesthisthreshold,thetransactioniscommittedautomaticallyandanewtransactionisstarted.Thevalueisspecifiedinbytes.

--rocksdb.intermediate-commit-count:ifthenumberofoperationsinatransactionreachesthisvalue,thetransactioniscommittedautomaticallyandanewtransactionisstarted.

--rocksdb.max-transaction-size:thisisanupperlimitforthetotalnumberofbytesofalloperationsinatransaction.Iftheoperationsinatransactionconsumemorethanthisthresholdvalue,thetransactionwillautomaticallyabortwitherror32("resourcelimitexceeded").

Itisalsopossibletooverridethesethresholdspertransaction.

Thefollowingknownissueswillberesolvedinfuturereleases:

theRocksDBengineisnotyetperformance-optimizedandpotentiallynotwellconfigured

collectionsforwhichageoindexispresentwillusecollection-levelwritelocksevenwiththeRocksDBengine.Readsfromthesecollectionscanstillbedoneinparallelbutnowrites

modifyingdocumentsinacollectionwithageoindexwillcausemultipleadditionalwritestoRocksDBformaintainingtheindexstructures

thenumberofdocumentsreportedforcollections(db.<collection>.count())maybeslightlywrongduringtransactionsifthereareparalleltransactionsongoingforthesamecollectionthatalsomodifythenumberofdocuments

theanyoperationtoprovidearandomdocumentfromacollectionissupportedbytheRocksDBenginebuttheoperationhasmuchhigheralgorithmiccomplexitythanintheMMFilesengine.Itisthereforediscouragedtocallitforcasesotherthanmanualinspectionofafewdocumentsinacollection

AQLqueriesintheclusterstillissueanextralockingHTTPrequestpershardthoughthiswouldnotbenecessaryfortheRocksDBengineinmostcases

Installer

KnownIssuesin3.2

Upgradingfrom3.1to3.2onWindowsrequirestheusertomanuallycopythedatabasedirectorytothenewlocationandrunanupgradeonthedatabase.PleaseconsulttheDocumentationfordetailedinstructions.

SystemIntegration

OnsomeLinuxsystemssystemdandsystemvmightreportthatthearangodbserviceisingoodconditionwhenitcouldnotbestarted.Inthiscasetheuserneedstocheck/var/log/arangodb3forfurtherinformationaboutthefailedstartup.

MacOSX

Storageengineisnotchangeableonanexistingdatabase.Currentlyonlytheinitialselectionofthestorageengineissupported.Inordertouseanotherstorageengine,youhavetodeleteyourArangoDBapplication(MacApplicationFolder)and/Users/<your_user_name>/Library/ArangoDBfolder.

OpenSSL1.1

ArangoDBhasbeentestedwithOpenSSL1.0onlyandwon'tbuildagainst1.1whencompilingonyourown.SeehereforhowtocompileonsystemsthatshipOpenSSL1.1bydefault.

KnownIssuesin3.2

AQLbreakingchangeincluster:TheSHORTEST_PATHstatementusingedge-collectionnamesinsteadofagraphnamenowrequirestoexplicitlynamethevertex-collectionnameswithintheAQLqueryinthecluster.ItcanbedonebyaddingWITH<name>atthebeginningofthequery.

Example:

Nowhastobe:

WITHvertices

Thischangeisduetoavoiddead-locksitationsinclusteredcase.Anerrorstatingtheaboveisincluded.

RESTAPIRemovedundocumentedinternalHTTPAPI:

PUT/_api/edgesThedocumentedGET/_api/edgesandtheundocumentedPOST/_api/edgesremainsunmodified.

changeundocumentedbehaviourincaseofinvalidrevisionidsinIf-MatchandIf-None-MatchheadersfromreturningHTTPstatuscode400(badrequest)toreturningHTTPstatuscode412(preconditionfailed).

theRESTAPIforfetchingthelistofcurrentlyrunningAQLqueriesandtheRESTAPIforfetchingthelistofslowAQLqueriesnowreturnanextrabindVarsattributewhichcontainsthebindparametersusedbythequeries.

ThisaffectsthereturnvaluesofthefollowingAPIendpoints:

GET/_api/query/currentGET/_api/query/slow

TheRESTAPIforretrievingindexes(GET/_api/index)nowreturnsthededuplicateattributeforeachindex

TheRESTAPIforcreatingindexes(POST/_api/index)nowacceptstheoptionaldeduplicateattribute

TheRESTAPIforexecutingaserver-sidetransaction(POST/_api/transaction)nowacceptstheoptionalattributes:maxTransactionSize,intermediateCommitCount,intermediateCommitSize

TheRESTAPIforcreatingacursor(POST/_api/cursor)nowacceptstheoptionalattributes:failOnWarning,maxTransactionSize,maxWarningCount,intermediateCommitCount,satelliteSyncWait,intermediateCommitSize.skipInaccessibleCollections

JavaScriptAPI

changeundocumentedbehaviourincaseofinvalidrevisionidsinJavaScriptdocumentoperationsfromreturningerrorcode1239("illegaldocumentrevision")toreturningerrorcode1200("conflict").

thecollection.getIndexes()functionnowreturnsthededuplicateattributeforeachindex

thecollection.ensureIndex()functionnowacceptstheoptionaldeduplicateattribute

JWTtokensissuedbythebuilt-inJWTsessionstoragenowcorrectlyspecifytheiatandexpvaluesinsecondsratherthanmillisecondsasspecifiedintheJSONWebTokenstandard.

Thismayresultinpreviouslyexpiredtokensusingmillisecondsbeingincorrectlyaccepted.ForthisreasonitisrecommendedtoreplacethesigningsecretorsetthenewmaxExpoptiontoareasonablevaluethatissmallerthantheoldestissuedexpirationtimestamp.

ForexamplesettingmaxExpto10**12wouldinvalidateallincorrectlyissuedtokensbefore9September2001withoutimpairingnewtokensuntiltheyear33658(atwhichpointthesetokensarehopefullynolongerrelevant).

ArangoDBrunninginstandalonemodewillcommitallservicesinthejavascript.app-pathtothedatabaseonstartup.ThismayresultinuninstalledservicesshowingupinArangoDBiftheywerenotproperlyremovedfromthefilesystem.

ArangoDBcoordinatorsinaclusternowperformaself-healingstepduringstartuptoensureinstalledservicesareconsistentaccrossallcoordinators.WerecommendbackingupyourservicesandconfigurationbeforeupgradingtoArangoDB3.2,especiallyifyouhavemadeuseofthedevelopmentmode.

Servicesinstalledbeforeupgradingto3.2(includingservicesinstalledonalphareleasesofArangoDB3.2)areNOTpickedupbythecoordinatorself-healingwatchdog.Thiscanbesolvedbyeitherupgrading/replacingtheseservicesorbyusingthe"commit"routeoftheFoxxservicemanagementHTTPAPI,whichcommitstheexactservicesinstalledonagivencoordinatortothecluster.Newserviceswillbepickedupautomatically.

TheformatusedbyFoxxtostoreinternalservicemetadatainthedatabasehasbeensimplifiedandexistingdocumentswillbeupdatedtothenewformat.Ifyouhavemadeanychangestothedatastoredinthe_appssystemcollection,youmaywishtoexportthesechangesastheywillbeoverwritten.

ThereisnowanofficialHTTPAPIformanagingservices.IfyouwerepreviouslyusinganyoftheundocumentedAPIsortheroutesusedbytheadministrativewebinterfacewehighlyrecommendmigratingtothenewAPI.TheoldundocumentedHTTPAPIformananagingservicesisdeprecatedandwillberemovedinafutureversionofArangoDB.

Althoughchangestothefilesystemoutsideofdevelopmentmodewerealreadystronglydiscouraged,thisisareminderthattheyarenolongersupported.Allfilesgeneratedbyservices(whetherbyasetupscriptorduringnormaloperationsuchasuploads)shouldeitherbestoredoutsidetheservicedirectoryorbeconsideredextremelyvolatile.

IntroduceddistinctionbetweenarangoUserandauthorizedinFoxxrequests.ClusterinternalrequestswillneverhaveanarangoUserbutareauthorized.InearlierversionsofArangoDBpartsofthestatisticswerenotaccessiblebythecoordinatorsbecausetheunderlyingFoxxservicecouldn'tauthorizetherequests.Itnowcorrectlychecksthenewreq.authorizedproperty.req.arangoUserstillworksasbefore.Endusersmayusethisnewpropertyaswelltoeasilycheckifarequestisauthorizedornotregardlessofaspecificuser.

Command-lineoptionschanged

--server.maximal-queue-sizeisnowanabsolutemaximum.Ifthequeueisfull,then503isreturned.Settingitto0means"nolimit".Thedefaultvalueforthisoptionisnow0.

thedefaultvaluefor--ssl.protocolhasbeenchangedfrom4(TLSv1)to5(TLSv1.2).

thestartupoptions--database.revision-cache-chunk-sizeand--database.revision-cache-target-sizearenowobsoleteanddonothing

thestartupoption--database.index-threadsoptionisnowobsolete

theoption--javascript.v8-contextsisnowanabsolutemaximum.TheservermaystartlessV8contextsforJavaScriptexecutionatstartup.IfatsomepointtheserverneedsmoreV8contextsitmaystartthemdynamically,untilthenumberofV8contextsreachesthevalueof--javascript.v8-contexts.

theminimumnumberofV8contextstocreateatstartupcanbeconfiguredviathenewstartupoption--javascript.v8-contexts-minimum.

addedcommand-lineoption--javascript.allow-admin-execute

Thisoptioncanbeusedtocontrolwhetheruser-definedJavaScriptcodeisallowedtobeexecutedonserverbysendingviaHTTPtotheAPIendpoint/_admin/executewithanauthenticateduseraccount.Thedefaultvalueisfalse,whichdisablestheexecutionofuser-definedcode.Thisisalsotherecommendedsettingforproduction.Intestenvironments,itmaybeconvenienttoturntheoptiononinordertosendarbitrarysetuporteardowncommandsforexecutionontheserver.

TheintroductionofthisoptionchangesthedefaultbehaviorofArangoDB3.2:3.2nowbydefaultdisablestheexecutionofJavaScriptcodeviathisAPI,whereasearlierversionsallowedit.Torestoretheoldbehavior,itisnecessarytosettheoptiontotrue.

UsersManagement

Itisnolongersupportedtoaccessthe_userscollecctioninanywaydirectly,exceptthroughtheofficial@arangodb/usersmoduleorthe_apit/usersRESTAPI.

Theaccesstothe_userscollectionfromoutsideofthearangodserverprocessisnowforbidden(Throughdrivers,arangoshortheRESTAPI).Foxxservicesarestillbeabletoaccessthe_userscollectionfornow,butthismightchangeinfutureminorreleases.

Theinternalformatofthedocumentsinthe_userscollectionhaschangedfrompreviousversions

The_queuescollectiononlyallowsread-onlyaccessfromoutsideofthearangodserverprocess.

Accessing_queuesisonlysupportedthroughtheofficial@arangodb/queuesmoduleforFoxxapps.

SmartGraphs

ArangoDB3.1addsafirstmajorenterpriseonlyfeaturecalledSmartGraphs.SmartGraphsformanadditiontothealreadyexistinggraphfeaturesandallowtoscalegraphsbeyondasinglemachinewhilekeepingalmostthesamequeryperformance.TheSmartGraphfeatureissuggestedforallgraphdatabaseusecasesthatrequireaclusterofdatabaseserversforwhateverreason.Youcaneitherhaveagraphthatistoolargetobestoredonasinglemachineonly.Oryoucanhaveasmallgraph,butatthesametimeneedadditionaldatawithhastobeshardedandyouwanttokeepalloftheminthesameenvirenment.Oryousimplyusetheclusterforhigh-availability.InalltheabovecasesSmartGraphswillsignificantlyincreasetheperformanceofgraphoperations.Formoredetailedinformationreadthismanualsection.

Dataformat

Theformatoftherevisionvaluesstoredinthe_revattributeofdocumentshasbeenchangedin3.1.Upto3.0theywerestringscontaininglargishdecimalnumbers.With3.1,revisionvaluesarestillstrings,butareactuallyencodedtimestampsofthecreationdateoftherevisionofthedocument.Thetimestampsareacquiredusingahybridlogicalclock(HLC)ontheDBserverthatholdstherevision(fortheconceptofahybridlogicalclockseethispaper).Seethismanualsectionfordetails.

ArangoDB>=3.1canArangoDB3.0databasedirectoriesandwillsimplycontinuetousetheold_revattributevalues.Newrevisionswillbewrittenwiththenewtimestamps.

ItishighlyrecommendedtobackupallyourdatabeforeloadingadatabasedirectorythatwaswrittenbyArangoDB<=3.0intoanArangoDB>=3.1.

CommunicationLayerArangoDBupto3.0usedlibevforthecommunicationlayer.ArangoDBstartingfrom3.1usesBoostASIO.

StartingwithArangoDB3.1webegintoprovidetheVelocyStreamProtocol(vst)asaadditiontotheestablishedhttpprotocol.

Afewoptionshavechangedconcerningcommunication,pleasecheckoutIncompatiblechangesin3.1.

ClusterForitsinternalclustercommunicationa(bundledversion)ofcurlisnowbeingused.Thisenablesasynchronousoperationthroughouttheclusterandshouldimprovegeneralperformanceslightly.

Authenticationisnowsupportedwithinthecluster.

Documentrevisionscache

TheArangoDBservernowprovidesanin-memorycacheforfrequentlyaccesseddocumentrevisions.Documentsthatareaccessedduringread/writeoperationsareloadedintotherevisionscacheautomatically,andsubsequentlyservedfromthere.

Thecachehasatotaltargetsize,whichcanbecontrolledwiththestartupoption--database.revision-cache-target-size.Oncethecachereachesthetargetsize,olderentriesmaybeevictedfromthecachetofreememory.Notethatthetargetsizecurrentlyisahighwatermarkthatwilltriggercachememorygarbagecollectionifexceeded.However,ifallcachechunksarestillinusewhenthehighwatermarkisreached,thecachemaystillgrowandallocatemorechunksuntilcacheentriesbecomeunusedandareallowedtobegarbage-collected.

WhatsNewin3.1

Thecacheismaintainedonaper-collectionbasis,thatis,memoryforthecacheisallocatedonaper-collectionbasisinchunks.Thesizeforthecachememorychunkscanbecontrolledviathestartupoption--database.revision-cache-chunk-size.Thedefaultvalueis4MBperchunk.Biggerchunksizesallowsavingmoredocumentsperchunk,whichcanleadtomoreefficientchunkallocationandlookups,butwillalsoleadtomemorywasteifmanychunksareallocatedandnotfullyused.Thelatterwillbethecaseifthereexistmanysmallcollectionswhichallallocatetheirownchunksbutnotfullyutilizethembecauseofthelownumberofdocuments.

Functionsadded

ThefollowingAQLfunctionshavebeenaddedin3.1:

OUTERSECTION(array1,array2,...,arrayn):returnsthevaluesthatoccuronlyonceacrossallarraysspecified.

DISTANCE(lat1,lon1,lat2,lon2):returnsthedistancebetweenthetwocoordinatesspecifiedby(lat1,lon1)and(lat2,lon2).Thedistanceiscalculatedusingthehaversineformula.

JSON_STRINGIFY(value):returnsaJSONstringrepresentationofthevalue.

JSON_PARSE(value):convertsaJSON-encodedstringintoaregularobject

Indexusageintraversals

3.1allowsAQLtraversalstouseotherindexesthanjusttheedgeindex.Traversalswithfiltersonedgescannowmakeuseofmorespecificindexes.Forexample,thequery

FORv,e,pIN2OUTBOUND@start@@edge

FILTERp.edges[0].foo=="bar"

RETURN[v,e,p]

mayuseahashindexon["_from","foo"]insteadoftheedgeindexonjust["_from"].

MaketheAQLqueryoptimizerinjectfilterconditionexpressionsreferredtobyvariablesduringfilterconditionaggregation.Forexample,inthefollowingquery

FORdocINcollection

LETcond1=(doc.value==1)

LETcond2=(doc.value==2)

FILTERcond1||cond2

RETURN{doc,cond1,cond2}

theoptimizerwillnowinjecttheconditionsforcond1andcond2intothefilterconditioncond1||cond2,expandingitto(doc.value==1)||(doc.value==2)andmakingtheseconditionsavailableforindexsearching.

Notethattheoptimizerpreviouslyalreadyinjectedsomeconditionsintootherconditions,butonlyifthevariablethatdefinedtheconditionwasnotusedelsewhere.Forexample,thefilterconditioninthequery

FORdocINcollection

LETcond=(doc.value==1)

FILTERcond

RETURN{doc}

alreadygotoptimizedbeforebecausecondwasonlyusedonceinthequeryandtheoptimizerdecidedtoinjectitintotheplacewhereitwasused.

Thisonlyworkedforvariablesthatwerereferredtoonceinthequery.Whenavariablewasusedmultipletimes,theconditionwasnotinjectedasinthefollowingquery

FORdocINcollection

WhatsNewin3.1

LETcond=(doc.value==1)

FILTERcond

RETURN{doc,cond}

3.1allowsusingthisconditionsothatthequerycanuseanindexondoc.value(ifsuchindexexists).

Theperformanceofthe[*]operatorwasimprovedforcasesinwhichthisoperatordidnotuseanyfilters,projectionsand/oroffset/limits.

TheAQLqueryexecutorcannowreportthetimerequiredforloadingandlockingthecollectionsusedinanAQLquery.Whenprofilingisenabled,itwillreportthetotalloadingandlockingtimeforthequeryintheloadingcollectionssub-attributeoftheextra.profilevalueoftheresult.TheloadingandlockingtimecanalsobeviewintheAQLqueryeditorinthewebinterface.

AuditLogAuditlogginghasbeenadded,seeAuditing.

ClienttoolsAddedoption--skip-linesforarangoimpThisallowsskippingthefirstfewlinesfromtheimportfileincasetheCSVorTSVimportareusedandsomeinitiallinesshouldbeskippedfromtheinput.

WebAdminInterfaceTheusabilityoftheAQLeditorsignificantlyimproved.InadditiontothestandardJSONoutput,theAQLEditorisnowabletorenderqueryresultsasagraphprevieworatable.FurthermoretheAQLeditordisplaysqueryprofilinginformation.

AddedanewGraphViewerinordertoexchangethetechnicallyobsoleteversion.ThenewGraphViewerisbasedonCanvasbutdoesalsoincludeafirstWebGLimplementation(limitedfunctionality-willchangeinthefuture).ThenewGraphVieweroffersasmoothwaytodiscoverandvisualizeyourgraphs.

Theshardviewinclustermodenowdisplaysaprogressindicatorwhilemovingshards.

AuthenticationUptoArangoDB3.0authenticationofclientrequestswasonlypossiblewithHTTPbasicauthentication.

Startingwith3.1itisnowpossibletoalsouseaJSONWebTokens(JWT)forauthenticatingincomingrequests.

FordetailschecktheHTTPauthenticationchapter.Bothauthenticationmethodsarevalidandwillbesupportedinthenearfuture.Usewhateversuitsyoubest.

GraphQL

ItisnoweasytogetstartedwithprovidingGraphQLAPIsinFoxx,seeFoxxGraphQL.

OAuth2

FoxxnowofficiallyprovidesamoduleforimplementingOAuth2clients,seeFoxxOAuth2.

Per-routemiddleware

WhatsNewin3.1

It'snowpossibletospecifymiddlewarefunctionsforaroutewhendefiningaroutehandler.Thesemiddlewarefunctionsonlyapplytothesinglerouteandsharetheroute'sparameterdefinitions.CheckouttheFoxxRouterdocumentationformoreinformation.

WhatsNewin3.1

CommunicationLayer

TheinternalcommicationlayerisnowbasedonBoostASIO.Afewoptionsregardingthreadsandcommunicationhavebeenchanged.

Therearenolongertwodifferentthreadspools(--scheduler.threadsand--server.threads).Theoption--scheduler.threadshasbeenremoved.Thenumberofthreadsisnowcontrolledbytheoption--server.threadsonly.Bydefault--server.threadsissettothenumberofhyper-cores.

Asaconsequenceofthechange,thefollowing(hidden)startupoptionshavebeenremoved:

--server.extra-threads

--server.aql-threads

--server.backend

--server.show-backends

--server.thread-affinity

ThebehavioroftheAQLarraycomparisonoperatorshaschangedforemptyarrays:

ALLandANYnowalwaysreturnfalsewhentheleft-handoperandisanemptyarray.Thebehaviorfornon-emptyarraysdoesnotchange:

[]ALL==1willreturnfalse[1]ALL==1willreturntrue[1,2]ALL==1willreturnfalse[2,2]ALL==1willreturnfalse[]ANY==1willreturnfalse[1]ANY==1willreturntrue[1,2]ANY==1willreturntrue[2,2]ANY==1willreturnfalse

NONEnowalwaysreturnstruewhentheleft-handoperandisanemptyarray.Thebehaviorfornon-emptyarraysdoesnotchange:

[]NONE==1willreturntrue[1]NONE==1willreturnfalse[1,2]NONE==1willreturnfalse[2,2]NONE==1willreturntrue

WITHinclustertraversalsisnowmandatoryinordertoavoiddeadlocks.

Dataformatchanges

TheattributemaximalSizehasbeenrenamedtojournalSizeincollectionmeta-datafiles("parameter.json").FilescontainingthemaximalSizeattributewillstillbepickedupcorrectlyfornot-yetadjustedcollections.

Theformatoftherevisionvaluesstoredinthe_revattributeofdocumentshasbeenchangedin3.1.Upto3.0theywerestringscontaininglargishdecimalnumbers.With3.1,revisionvaluesarestillstrings,butareactuallyencodedtimestampsofthecreationdateoftherevisionofthedocument.Thetimestampsareacquiredusingahybridlogicalclock(HLC)ontheDBserverthatholdstherevision(fortheconceptofahybridlogicalclockseethispaper).Seethismanualsectionfordetails.

ArangoDB>=3.1canArangoDB3.0databasedirectoriesandwillsimplycontinuetousetheold_revattributevalues.Newrevisionswillbewrittenwiththenewtimestamps.

ItishighlyrecommendedtobackupallyourdatabeforeloadingadatabasedirectorythatwaswrittenbyArangoDB<=3.0intoanArangoDB>=3.1.

Tochangeallyourold_revattributesintonewstyletimestampsyouhavetousearangodumptodumpalldataout(usingArangoDB3.0),andusearangorestoreintothenewArangoDB3.1,whichisthesafestwaytoupgrade.

Thechangealsoaffectsthereturnformatof_revvaluesandotherrevisionvaluesinHTTPAPIs(seebelow).

HTTPAPIchanges

APIsadded

ThefollowingHTTPRESTAPIshavebeenaddedforonlinelogleveladjustmentoftheserver:

GET/_admin/log/levelreturnsthecurrentloglevelsettingsPUT/_admin/log/levelmodifiesthecurrentloglevelsettings

APIschanged

thefollowingRESTAPIsthatreturnrevisionidsnowmakeuseofthenewrevisionidformatintroducedin3.1.Allrevisionidsreturnedwillbestringsasin3.0,buthaveadifferentinternalformat.

ThefollowingAPIsareaffected:

GET/_api/collection/{collection}/checksum:revisionattributeGET/_api/collection/{collection}/revision:revisionattributeallotherAPIsthatreturndocuments,whichmayincludethedocuments'_revattribute

Clientapplicationsshouldnottrytointerprettheinternalsofrevisionvalues,butonlyuserevisionvaluesforcheckingwhethertworevisionstringsareidentical.

thereplicationRESTAPIswillnowusetheattributenamejournalSizeinsteadofmaximalSizewhenreturninginformationaboutcollections.

thedefaultvalueforkeepNullhasbeenchangedfromfalsetotrueforthefollowingpartialupdateoperationsforverticesandedgesin/_api/gharial:

PATCH/_api/gharial/{graph}/vertex/{collection}/{key}PATCH/_api/gharial/{graph}/edge/{collection}/{key}

ThevalueforkeepNullcanstillbesetexplicitlytofalsebysettingtheURLparameterkeepNulltoavalueoffalse.

theRESTAPIfordroppingcollections(DELETE/_api/collection)nowacceptsanoptionalquerystringparameterisSystem,whichcansettotrueinordertodropsystemcollections.Iftheparameterisnotsetornotsettotrue,theRESTAPIwillrefusetodropsystemcollections.InpreviousversionsofArangoDB,theisSystemparameterdidnotexist,andtherewasnodistinctionbetweensystemandnon-systemcollectionswhendroppingcollections.

theRESTAPIforretrievingAQLqueryresults(POST/_api/cursor)willnowreturnanadditionalsub-attributeloadingcollectionsthatwillcontainthetotaltimerequiredforloadingandlockingcollectionsduringtheAQLquerywhenprofilingisenabled.Theattributecanbefoundintheextraresultattributeinsub-attributeloadingcollections.Theattributewillonlybesetifprofilingwasenabledforthequery.

theRESTAPIforretrievingAQLqueryresults(POST/_api/cursor)willnowaccepttheoptionalattributememoryLimit.

FoxxTestingTheQUnitinterfacetoMochahasbeenremoved.Thisaffectsthebehaviourofthesuite,test,before,after,beforeEachandafterEachfunctionsinFoxxtestsuites.ThesuiteandtestfunctionsarenowprovidedbytheTDDinterface.Thebefore,after,beforeEachandafterEachfunctionsarenowprovidedbytheBDDinterface.

Thisshouldnotcauseanyproblemswithexistingtestsbutmayresultinfailuresintestcasesthatpreviouslypassedforthewrongreasons.Specificallytheexecutionorderofthebefore,after,etcfunctionsnowfollowstheintendedorderandisnolongerarbitrary.

FordetailsontheexpectedbehaviourofthesefunctionsseethetestingchapterintheFoxxdocumentation.

Internaldataformatchanges

ArangoDBnowusesVelocyPackforstoringdocuments,queryresultsandtemporarilycomputedvalues.Usingasingledataformatremovedtheneedforsomedataconversionsinthecorethatslowedoperationsdownpreviously.

TheVelocyPackformatisalsoquitecompact,andreducesstoragespacerequirementsfor"small"valuessuchasboolean,integers,shortstrings.ThiscanspeedupseveraloperationsinsideAQLqueries.

VelocyPackdocumententriesstoredondiskarealsoself-contained,inthesensethateachstoreddocumentwillcontainallofitsdatatypeandattributenamedescriptions.Whilethismayrequireabitmorespaceforstoringthedocuments,itremovestheoverheadoffetchingattributenamesanddocumentlayoutfromsharedstructuresasinpreviousversionsofArangoDB.Italsosimplifiesthecodepathsforstoringandreadingdocuments.

AQLimprovements

Syntaximprovements

LIKEstring-comparisonoperator

AQLnowprovidesaLIKEoperatorandcanbeusedtocomparestringslikethis,forexampleinsidefilterconditions:

valueLIKEsearch

ThischangemakesLIKEanAQLkeyword.UsingLIKEasanattributeorcollectionnameinAQLthusrequiresquotingthenamefromnowon.

TheLIKEoperatoriscurrentlyimplementedbycallingthealreadyexistingAQLfunctionLIKE,whichalsoremainsoperationalin3.0.UsetheLIKEfunctionincaseyouwanttosearchcase-insensitive(optionalparameter),astheLIKEoperatoralwayscomparescase-sensitive.

AQLarraycomparisonoperators

AllAQLcomparisonoperatorsnowalsoexistinanarrayvariant.Inthearrayvariant,theoperatorisprecededwithoneofthekeywordsALL,ANYorNONE.Usingoneofthesekeywordschangestheoperatorbehaviortoexecutethecomparisonoperationforall,any,ornoneofitslefthandargumentvalues.Itisthereforeexpectedthatthelefthandargumentofanarrayoperatorisanarray.

Examples:

[1,2,3]ALLIN[2,3,4]//false

[1,2,3]ALLIN[1,2,3]//true

[1,2,3]NONEIN[3]//false

[1,2,3]NONEIN[23,42]//true

[1,2,3]ANYIN[4,5,6]//false

[1,2,3]ANYIN[1,42]//true

[1,2,3]ANY==2//true

[1,2,3]ANY==4//false

[1,2,3]ANY>0//true

[1,2,3]ANY<=1//true

[1,2,3]NONE<99//false

[1,2,3]NONE>10//true

[1,2,3]ALL>2//false

[1,2,3]ALL>0//true

[1,2,3]ALL>=3//false

["foo","bar"]ALL!="moo"//true

WhatsNewin3.0

["foo","bar"]NONE=="bar"//false

["foo","bar"]ANY=="foo"//true

Regularexpressionstring-comparisonoperators

AQLnowsupportstheoperators=~and!~fortestingstringsagainstregularexpressions.=~testsifastringvaluematchesaregularexpression,and!~testsifastringvaluedoesnotmatcharegularexpression.

Thetwooperatorsexpecttheirleft-handoperandstobestrings,andtheirright-handoperandstobestringscontainingvalidregularexpressionsasspecifiedbelow.

Theregularexpressionsmayconsistofliteralcharactersandthefollowingcharactersandsequences:

.–thedotmatchesanysinglecharacterexceptlineterminators.Toincludelineterminators,use[\s\S]insteadtosimulate.withDOTALLflag.\d–matchesasingledigit,equivalentto[0-9]\s–matchesasinglewhitespacecharacter\S–matchesasinglenon-whitespacecharacter\t–matchesatabcharacter\r–matchesacarriagereturn\n–matchesaline-feedcharacter[xyz]–setofcharacters.matchesanyoftheenclosedcharacters(i.e.x,yorzinthiscase[^xyz]–negatedsetofcharacters.matchesanyothercharacterthantheenclosedones(i.e.anythingbutx,yorzinthiscase)[x-z]–rangeofcharacters.Matchesanyofthecharactersinthespecifiedrange,e.g.[0-9A-F]tomatchanycharacterin0123456789ABCDEF[^x-z]–negatedrangeofcharacters.Matchesanyothercharacterthantheonesspecifiedintherange(xyz)–definesandmatchesapatterngroup(x|y)–matcheseitherxory –matchesthebeginningofthestring(e.g. xyz)$–matchestheendofthestring(e.g.xyz$)

Notethatthecharacters.,*,?,[,],(,),{,}, ,and$haveaspecialmeaninginregularexpressionsandmayneedtobeescapedusingabackslash(\\).Aliteralbackslashshouldalsobeescapedusinganotherbackslash,i.e.\\\\.

Charactersandsequencesmayoptionallyberepeatedusingthefollowingquantifiers:

x*–matcheszeroormoreoccurrencesofxx+–matchesoneormoreoccurrencesofxx?–matchesoneorzerooccurrencesofxx{y}–matchesexactlyyoccurrencesofxx{y,z}–matchesbetweenyandzoccurrencesofxx{y,}–matchesatleastyoccurencesofx

Enclosingidentifiersinforwardticks

AQLidentifierscannowoptionallybeenclosedinforwardticksinadditiontousingbackwardticks.ThisallowsconvenientwritingofAQLqueriesinJavaScripttemplatestrings(whicharedelimitedwithbackticksthemselves),e.g.

varq=`FORdocIN´collection´RETURNdoc.´name´`;

Functionsadded

REGEX_TEST(value,regex):testswhetherthestringvaluematchestheregularexpressionspecifiedinregex.Returnstrueifitmatches,andfalseotherwise.

Thesyntaxforregularexpressionsisthesameasfortheregularexpressionoperators=~and!~.

WhatsNewin3.0

HASH(value):Calculatesahashvalueforvalue.valueisnotrequiredtobeastring,butcanhaveanydatatype.Thecalculatedhashvaluewilltakethedatatypeofvalueintoaccount,soforexamplethenumber1andthestring"1" willhavedifferenthashvalues.Forarraysthehashvalueswillbecrearedifthearrayscontainexactlythesamevalues(includingvaluetypes)inthesameorder.Forobjectsthesamehashvalueswillbecreatediftheobjectshaveexactlythesameattributenamesandvalues(includingvaluetypes).Theorderinwhichattributesappearinsideobjectsisnotimportantforhashing.Thehashvaluereturnedbythisfunctionisanumber.ThehashalgorithmisnotguaranteedtoremainthesameinfutureversionsofArangoDB.Thehashvaluesshouldthereforebeusedonlyfortemporarycalculations,e.g.tocompareiftwodocumentsarethesame,orforgroupingvaluesinqueries.

TYPENAME(value):Returnsthedatatypenameofvalue.Thedatatypenamecanbeeithernull,bool,number,string,arrayorobject.

LOG(value):Returnsthenaturallogarithmofvalue.ThebaseisEuler'sconstant(2.71828...).

LOG2(value):Returnsthebase2logarithmofvalue.

LOG10(value):Returnsthebase10logarithmofvalue.

EXP(value):ReturnsEuler'sconstant(2.71828...)raisedtothepowerofvalue.

EXP2(value):Returns2raisedtothepowerofvalue.

SIN(value):Returnsthesineofvalue.

COS(value):Returnsthecosineofvalue.

TAN(value):Returnsthetangentofvalue.

ASIN(value):Returnsthearcsineofvalue.

ACOS(value):Returnsthearccosineofvalue.

ATAN(value):Returnsthearctangentofvalue.

ATAN2(y,x):Returnsthearctangentofthequotientofyandx.

RADIANS(value):Returnstheangleconvertedfromdegreestoradians.

DEGREES(value):Returnstheangleconvertedfromradianstodegrees.

"inline-subqueries"rule

TheAQLoptimizerrule"inline-subqueries"hasbeenadded.ThisrulecanpulloutcertainsubqueriesthatareusedasanoperandtoaFORlooponelevelhigher,eliminatingthesubquerycompletely.Thisreducescomplexityofthequery'sexecutionplanandwilllikelyenablefurtheroptimizations.Forexample,thequery

FORiIN(

FORjIN[1,2,3]

RETURNj

RETURNi

willbetransformedbytheruleto:

FORiIN[1,2,3]

RETURNi

Thequery

FORnameIN(

FORdocIN_users

FILTERdoc.status==1

RETURNdoc.name

LIMIT2

WhatsNewin3.0

RETURNname

willbetransformedinto

FORtmpIN_users

FILTERtmp.status==1

LIMIT2

RETURNtmp.name

TherulewillonlyfirewhenthesubqueryisusedasanoperandtoaFORloop,andifthesubquerydoesnotcontainaCOLLECTwithanINTOvariable.

"remove-unnecessary-calculations"rule

TheAQLoptimizerrule"remove-unnecessary-calculations"nowfiresinmorecasesthaninpreviousversions.Thisruleremovescalculationsfromexecutionplans,andbyhavinglesscalculationsdone,aquerymayexecutefasterorrequireslessmemory.

Therulewillnowremovecalculationsthatareusedexactlyonceinotherexpressions(e.g.LETa=docRETURNa.value)andcalculations,orcalculationsthatarejustreferencestoothervariables(e.g.LETa=b).

"optimize-traversals"rule

TheAQLoptimizerrule"merge-traversal-filter"wasrenamedto"optimize-traversals".TherulewillremoveunusededgeandpathresultvariablesfromthetraversalincasetheyarespecifiedintheFORsectionofthetraversal,butnotreferencedlaterinthequery.Thissavesconstructingedgesandpathsresultsthatarenotusedlater.

AQLnowusesVelocyPackinternallyforstoringintermediatevalues.Formanyvaluetypesitcannowgetawaywithoutextramemoryallocationsandlessinternalconversions.ValuescanbepassedintointernalAQLfunctionswithoutcopyingthem.ThiscanleadtoreducedqueryexecutiontimesforqueriesthatuseC++-basedAQLfunctions.

"replace-or-with-in"and"use-index-for-sort"rules

Theserulesnowfireinsomeadditionalcases,whichallowssimplifyingindexlookupconditionsandremovingSortNodesfromexecutionplans.

Clusterstatemanagement

Thecluster'sinternalstateinformationisnowalsomanagedbyArangoDBinstances.Earlierversionsreliedonthirdpartysoftwarebeinginstalledforthestoringtheclusterstate.ThestateismanagedbydedicatedArangoDBinstances,whichcanbestartedinaspecialagencymode.Theseinstancescanoperateinadistributedfashion.Theywillautomaticallyelectoneofthemtobecometheirleader,beingresponsibileforstoringthestatechangessentfromserversinthecluster.Theotherinstanceswillautomaticallyfollowtheleaderandwilltransparentlystandinshoulditbecomeunavailable.Theagencyinstancesarealsoself-organizing:theywillcontinuouslyprobeeachotherandre-electleaders.Thecommunicationbetweentheagencyinstancesusetheconsensus-basedRAFTprotocol.

TheoperationsforstoringandretrievingclusterstateinformationarenowmuchlessexpensivefromanArangoDBclusternodeperspective,whichinturnallowsforfasterclusteroperationsthatneedtofetchorupdatetheoverallclusterstate.

_fromand _toattributesofedgesareupdatableandusableinindexes

InArangoDBpriorto3.0theattributes_fromand_toofedgesweretreatedspeciallywhenloadingorstoringedges.Thatspecialhandlingledtotheseattributesbeingnotasflexibleasregulardocumentattributes.Forexample,the_fromand_toattributevaluesofanexistingedgecouldnotbeupdatedoncetheedgewascreated.Nowthisispossibleviathesingle-documentAPIsandviaAQL.

Additionally,the_fromand_toattributescouldnotbeindexedinuser-definedindexes,e.g.tomakeeachcombinationof_fromand_tounique.Finally,as_fromand_toreferencedthelinkedcollectionsbycollectionidandnotbycollectionname,theirmeaningbecameunclearonceareferencedcollectionwasdropped.Thecollectionidstoredinedgesthenbecameunusable,andwhenaccessing

WhatsNewin3.0

suchedgethecollectionnamepartofitwasalwaystranslatedto_undefined.

InArangoDB3.0,the_fromand_tovaluesofedgesaresavedasregularstrings.Thisallowsusing_fromand_toinuser-definedindexes.Additionally,thisallowstoupdatethe_fromand_tovaluesofexistingedges.Furthermore,collectionsreferencedby_fromand_tovaluesmaybedroppedandre-createdlater.Any_fromand_tovaluesofedgespointingtosuchdroppedcollectionareunaffectedbythedropoperationnow.

UnifiedAPIsforCRUDoperations

TheCRUDAPIsfordocumentsandedgehavebeenunified.EdgescannowbeinsertedandmodifiedviathesameAPIsasdocuments._fromand_toattributevaluescanbepassedasregulardocumentattributesnow:

db.myedges.insert({_from:"myvertices/some",_to:"myvertices/other",...});

Passing_fromand_toseparatelyasitwasrequiredinearlierversionsisnotnecessaryanymorebutwillstillwork:

db.myedges.insert("myvertices/some","myvertices/other",{...});

TheCRUDoperationsnowalsosupportbatchvariantsthatworksonarraysofdocuments/edges,e.g.

db.myedges.insert([

{_from:"myvertices/some",_to:"myvertices/other",...},

{_from:"myvertices/who",_to:"myvertices/friend",...},

{_from:"myvertices/one",_to:"myvertices/two",...},

ThebatchvariantsarealsoavailableinArangoDB'sHTTPAPI.Theycanbeusedtomoreefficientlycarryoutoperationswithmultipledocumentsthantheirsingle-documentequivalents,whichrequiredoneHTTPrequestperoperation.Withthebatchoperations,theHTTPrequest/responseoverheadcanbeamortizedacrossmultipleoperations.

Persistentindexes

ArangoDB3.0providesanexperimentalpersistentindexfeature.Persistentindexesstoretheindexvaluesondiskinsteadofin-memoryonly.Thismeanstheindexesdonotneedtoberebuiltin-memorywhenacollectionisloadedorreloaded,whichshouldimprovecollectionloadingtimes.

ThepersistentindexesinArangoDBarebasedontheRocksDBengine.Tocreateapersistentindexforacollection,createanindexoftype"rocksdb"asfollows:

db.mycollection.ensureIndex({type:"rocksdb",fields:["fieldname"]});

Thepersistentindexesaresorted,sotheyallowequalitylookupsandrangequeries.Notethatthefeatureisstillhighlyexperimentalandhassomeknowndeficiencies.Itwillbefinalizeduntilthereleaseofthe3.0stableversion.

UpgradedV8version

TheV8enginethatisusedinsideArangoDBtoexecuteJavaScriptcodehasbeenupgradedfromversion4.3.61to5.0.71.39.ThenewversionmakesseveralmoreES6featuresavailablebydefault,including

arrowfunctionscomputedpropertynamesrestparametersarraydestructuringnumericandobjectliterals

WhatsNewin3.0

WebAdminInterface

TheArangoDB3.0webinterfaceissignificantlyimproved.Itnowcomeswithamoreresponsivedesign,makingiteasiertouseondifferentdevices.Navigationandmenushavebeensimplified,andrelateditemshavebeenregroupedtostayclosertogetherandallowtighterworkflows.

TheAQLqueryeditorisnowmucheasiertouse.Multiplequeriescanbestartedandtrackedinparallel,whileresultsofearlierqueriesarestillpreserved.Queriesstillrunningcanbecanceleddirectlyfromtheeditor.TheAQLqueryeditornowallowstheusageofbindparameterstoo,andprovidesahelperforfindingcollectionnames,AQLfunctionnamesandkeywordsquickly.

Thewebinterfacenowkeepstrackofwhethertheserverisofflineandofwhichserver-sideoperationshavebeenstartedandarestillrunning.Itnowremainsusablewhilesuchlonger-runningoperationsareongoing.Italsokeepsmorestateaboutuser'schoices(e.g.windowssizes,whetherthetreeorthecodeviewwaslastusedinthedocumenteditor).

Clusterstatisticsarenowintegratedintothewebinterfaceaswell.Additionally,amenuitem"Helpus"hasbeenaddedtoeasilyprovidetheArangoDBteamfeedbackabouttheproduct.

Thefrontendmaynowbemountedbehindareverseproxyonadifferentpath.ForthistoworktheproxyshouldsendaX-Script-Nameheadercontainingthepath.

Abackendconfigurationforhaproxymightlooklikethis:

reqaddX-Script-Name:\/arangodb

Thefrontendwillrecognizethesubpathandproduceappropriatelinks.ArangoDBwillonlyacceptpathsfromtrustedfrontendproxies.Trustedproxiesmaybeaddedonstartup:

--frontend.proxy-request-checktrue--frontend.trusted-proxy192.168.1.117

--frontend.trusted-proxymaybeanyaddressornetmask.

Todisablethecheckandblindlyacceptanyx-script-nameset--frontend.proxy-request-checktofalse.

Foxximprovements

TheFoxxframeworkhasbeencompletelyrewrittenfor3.0withanew,simplerandmorefamiliarAPI.Themostnotablechangesare:

Legacymodefor2.8services

Stuckwitholdcode?Youcancontinueusingyour2.8-compatibleFoxxserviceswith3.0byadding"engines":{"arangodb":"^2.8.0"}(orsimilarversionrangesthatexclude3.0andup)totheservicemanifest.

Nomoreglobalvariablesandmagicalcomments

TheapplicationContextisnowmodule.context.Insteadofmagicalcommentsjustusethesummaryanddescriptionmethodstodocumentyourroutes.

RepositoryandModelhavebeenremoved

InsteadofrepositoriesjustuseArangoDBcollectionsdirectly.Forvalidationsimplyusethejoischemas(butwrappedinjoi.object())thatpreviouslylivedinsidethemodel.CollectionsandqueriesreturnplainJavaScriptobjects.

Controllershavebeenreplacedwithnestablerouters

Createrouterswithrequire('@arangodb/foxx/router')(),attachthemtoyourservicewithmodule.context.use(router).Becauseroutersarenolongermountedautomagically,youcanexportandimportthemlikeanyotherobject.Userouter.use('/path',subRouter)tonestroutersasdeeplyasyouwant.

Routescanbenamedandreversed

NomorememorizingURLs:addanametoyourroutelikerouter.get('/hello/:name',function(){...},'hello')andredirecttothefullURLwithres.redirect(req.resolve('hello',{name:'world'})).

WhatsNewin3.0

Simplerexpress-likemiddleware

Ifyoualreadyknowexpress,thisshouldbefamiliar.Here'sarequestloggerinthreelinesofcode:

varstart=Date.now();

try{next();}

finally{console.log(`${req.method}${req.url}${res.statusCode}${Date.now()-start}ms`);}

Sessionsandauthwithoutdependencies

Tomakeiteasiertogetstarted,thefunctionalitypreviouslyprovidedbythesimple-auth,oauth2,sessions-localandsessions-jwtserviceshavebeenmovedintoFoxxasthe@arangodb/foxx/auth,@arangodb/foxx/oauth2and@arangodb/foxx/sessionsmodules.

Logging

ArangoDB'sloggingisnowgroupedintotopics.Thelogverbosityandoutputfilescanbeadjustedperlogtopic.Forexample

--log.levelstartup=trace--log.levelqueries=trace--log.levelinfo

willlogmessagesconcerningstartupattracelevel,AQLqueriesattracelevelandeverythingelseatinfolevel.--log.levelcanbespecifiedmultipletimesatstartup,forasmanytopicsasneeded.

Somerelevantlogtopicsavailablein3.0are:

collector:informationabouttheWALcollector'sstatecompactor:informationaboutthecollectiondatafilecompactordatafiles:datafile-relatedoperationsmmap:informationaboutmemory-mappingoperations(includingmsync)queries:executedAQLqueries,slowqueriesreplication:replication-relatedinforequests:HTTPrequestsstartup:informationaboutserverstartupandshutdownthreads:informationaboutthreads

Thisalsoallowsdirectinglogoutputtodifferentfilesbasedontopics.Forexample,tologallAQLqueriestoafile"queries.log"onecanusetheoptions:

--log.levelqueries=trace--log.outputqueries=file:///path/to/queries.log

ToadditionallylogHTTPrequesttoafilenamed"requests.log"addtheoptions:

--log.levelrequests=info--log.outputrequests=file:///path/to/requests.log

Buildsystem

ArangoDBnowusesthecross-platformbuildsystemCMakeforallitsbuilds.Previousversionsusedtwodifferentbuildsystems,makingdevelopmentandcontributionsharderthannecessary.Nowthebuildsystemisunified,andalltargets(Linux,Windows,MacOS)arebuiltfromthesamesetofbuildinstructions.

Documentation

Thedocumentationhasbeenenhancedandre-organizedtobemoreintuitive.

WhatsNewin3.0

AnewintroductionforbeginnersshouldbringyouuptospeedwithArangoDBinlessthananhour.Additionaltopicshavebeenintroducedandwillbeextendedwithupcomingreleases.

ThetopicsAQLandHTTPAPIarenowseparatedfromthemanualforbettersearchabilityandlessconfusion.Aversionswitchermakesiteasiertojumptotheversionofthedocsyouareinterestedin.

WhatsNewin3.0

Buildsystem

BuildingArangoDB3.0fromsourcenowrequiresCMake.

Thepre-3.0buildsystemusedaconfigure-basedapproach.ThestepstobuildArangoDB2.8fromsourcecodewere:

makesetup

./configure<options>

Thesestepswillnotworkanymore,asArangoDB3.0doesnotcomewithaconfigurescript.

Tobuild3.0onLinux,createaseparatebuilddirectoryfirst:

mkdir-pbuild

andthencreatetheinitialbuildscriptsonceusingCMake:

(cdbuild&&cmake<options>..)

Theabovecommandwillconfigurethebuildandcheckfortherequireddependencies.Ifeverythingworkswelltheactualbuildcanbestartedwith

(cdbuild&&make)

ThebinariesfortheArangoDBserverandallclienttoolswillthenbecreatedinsidethebuilddirectory.TostartArangoDBlocallyfromthebuilddirectory,use

build/bin/arangod<options>

Datafilesanddatafilenames

ArangoDB3.0usesanewVelocyPack-basedformatforstoringdatainWALlogfilesandcollectiondatafiles.ThefileformatisnotcompatiblewiththefilesusedinpriorversionsofArangoDB.ThatmeansdatafileswrittenbyArangoDB3.0cannotbeusedinearlierversionsandviceversa.

Thepatternforcollectiondirectorynameswaschangedin3.0toincludearandomidcomponentattheend.Thenewpatterniscollection-<id>-<random>,where<id>isthecollectionidand<random>isarandomnumber.PreviousversionsofArangoDBusedapatterncollection-<id>withouttherandomnumber.

UserManagementUnlikeArangoDB2.x,ArangoDB3.0usersarenowseparatedfromdatabases,andyoucangrantoneormoredatabasepermissionstoauser.

IfyouwanttomimicthebehaviorofArangoDB,youshouldnameyouruserslikeusername@dbname.

Usersthatcanaccessthe_systemdatabaseareallowedtomanageusersandpermissionsforalldatabases.

Edgesandedgesattributes

InArangoDBpriorto3.0theattributes_fromand_toofedgesweretreatedspeciallywhenloadingorstoringedges.Thatspecialhandlingledtotheseattributesbeingnotasflexibleasregulardocumentattributes.Forexample,the_fromand_toattributevaluesofanexistingedgecouldnotbeupdatedoncetheedgewascreated.Additionally,the_fromand_toattributescouldnotbeindexedinuser-definedindexes,e.g.tomakeeachcombinationof_fromand_tounique.Finally,as_fromand_toreferencedthelinkedcollectionsbycollectionidandnotbycollectionname,theirmeaningbecameunclearonceareferencedcollectionwasdropped.Thecollectionidstoredinedgesthenbecameunusable,andwhenaccessingsuchedgethecollectionnamepartofitwasalwaystranslatedto_undefined.

InArangoDB3.0,the_fromand_tovaluesofedgesaresavedasregularstrings.Thisallowsusing_fromand_toinuser-definedindexes.Additionallythisallowsupdatingthe_fromand_tovaluesofexistingedges.Furthermore,collectionsreferencedby_fromand_tovaluesmaybedroppedandre-createdlater.Any_fromand_tovaluesofedgespointingtosuchdroppedcollectionareunaffectedbythedropoperationnow.Alsonotethatrenamingthecollectionreferencedin_fromand_toinArangoDB2.8alsorelinkedtheedges.In3.0theedgesareNOTautomaticallyrelinkedtothenewcollectionanymore.

DocumentsDocuments(incontrasttoedges)cannotcontaintheattributes_fromor_toonthemainlevelinArangoDB3.0.Theseattributeswillbeautomaticallyremovedwhensavingdocuments(i.e.non-edges)._fromand_tocanbestillusedinsub-objectsinsidedocuments.

The_fromand_toattributeswillofcoursebepreservedandarestillrequiredwhensavingedges.

Edgeshandling

WhenupdatingorreplacingedgesviaAQL,anymodificationstothe_fromand_toattributesofedgeswereignoredbypreviousversionsofArangoDB,withoutsignalinganyerrors.Thiswasduetothe_fromand_toattributesbeingimmutableinearlierversionsofArangoDB.

From3.0on,the_fromand_toattributesofedgesaremutable,soanyAQLqueriesthatmodifythe_fromor_toattributevaluesofedgeswillattempttoactuallychangetheseattributes.Clientsshouldbeawareofthischangeandshouldreviewtheirqueriesthatmodifyedgestoruleoutunintendedside-effects.

Additionally,whencompletelyreplacingthedataofexistingedgesviatheAQLREPLACEoperation,itisnowrequiredtospecifyvaluesforthe_fromand_toattributes,asREPLACErequirestheentirenewdocumenttobespecified.Ifeither_fromor_toaremissingfromthereplacementdocument,anREPLACEoperationwillfail.

Graphfunctions

Inversion3.0allformergraphrelatedfunctionshavebeenremovedfromAQLtobereplacedbynativeAQLconstructs.Theseconstructsallowformorefine-grainedfilteringonseveralgraphlevels.AlsothisallowstheAQLoptimizertoautomaticallyimprovethesequeriesbyenhancingthemwithappropriateindexes.Wehavecreatedrecipestoupgradefrom2.8to3.0whenusingthesefunctions.

Thefunctions:

GRAPH_COMMON_NEIGHBORSGRAPH_COMMON_PROPERTIESGRAPH_DISTANCE_TOGRAPH_EDGESGRAPH_NEIGHBORSGRAPH_TRAVERSALGRAPH_TRAVERSAL_TREEGRAPH_SHORTEST_PATHGRAPH_PATHSGRAPH_VERTICES

arecoveredinMigratingGRAPH_*Functionsfrom2.8orearlierto3.0

GRAPH_ABSOLUTE_BETWEENNESSGRAPH_ABSOLUTE_CLOSENESSGRAPH_ABSOLUTE_ECCENTRICITYGRAPH_BETWEENNESSGRAPH_CLOSENESSGRAPH_DIAMETERGRAPH_ECCENTRICITYGRAPH_RADIUS

arecoveredinMigratingGRAPH_*Measurementsfrom2.8orearlierto3.0

EDGESNEIGHBORSPATHSTRAVERSALTRAVERSAL_TREE

arecoveredinMigratinganonymousgraphfunctionsfrom2.8orearlierto3.0

Typecastingfunctions

ThetypecastingappliedbytheTO_NUMBER()AQLfunctionhaschangedasfollows:

stringvaluesthatdonotcontainavalidnumericvaluearenowconvertedtothenumber0.InpreviousversionsofArangoDBsuchstringvalueswereconvertedtothevaluenull.arrayvalueswithmorethan1memberarenowconvertedtothenumber0.InpreviousversionsofArangoDBsucharrayswereconvertedtothevaluenull.objects/documentsarenowconvertedtothenumber0.InpreviousversionsofArangoDBobjects/documentswereconvertedtothevaluenull.

Additionally,theTO_STRING()AQLfunctionnowconvertsnullvaluesintoanemptystring("")insteadofthestring"null",whichismoreinlinewithLENGTH(null)returning0andnot4sincev2.6.

TheoutputofTO_STRING()hasalsochangedforarraysandobjectsasfollows:

arraysarenowconvertedintotheirJSON-stringifyequivalents,e.g.

[]isnowconvertedto[][1,2,3]isnowconvertedto[1,2,3]["test",1,2]isnowconvertedto["test",1,2]`

PreviousversionsofArangoDBconvertedarrayswithnomembersintotheemptystring,andnon-emptyarraysintoacomma-separatedlistofmembervalues,withoutthesurroundingangularbrackets.Additionally,stringarraymemberswerenotenclosedinquotesintheresultstring:

[]wasconvertedto``[1,2,3]wasconvertedto1,2,3["test",1,2]wasconvertedtotest,1,2`

objectsarenowconvertedtotheirJSON-stringifyequivalents,e.g.

{}isconvertedto{}{a:1,b:2}isconvertedto{"a":1,"b":2}{"test":"foobar"}isconvertedto{"test":"foobar"}

PreviousversionsofArangoDBalwaysconvertedobjectsintothestring[objectObject]

ThischangealsoaffectsotherpartsinAQLthatusedTO_STRING()toimplicitlycastoperandstostrings.ItalsoaffectstheAQLfunctionsCONCAT()andCONCAT_SEPARATOR()whichtreatedarrayvaluesdifferently.PreviousversionsofArangoDBautomaticallyflattenedarrayvaluesinthefirstlevelofthearray,e.g.CONCAT([1,2,3,[4,5,6]])produced1,2,3,4,5,6.Nowthiswillproduce[1,2,3,[4,5,6]].Toflattenarraymembersonthetoplevel,youcannowusethemoreexplicitCONCAT(FLATTEN([1,2,3,[4,5,6]],1)).

Arithmeticoperators

AsthearithmeticoperationsinAQLimplicitlyconverttheiroperandstonumericvaluesusingTO_NUMBER(),theircastingbehaviorhasalsochangedasdescribedabove.

Someexamplesofthechangedbehavior:

"foo"+1produces1now.Inpreviousversionsthisproducednull.[1,2]+1produces1.Inpreviousversionsthisproducednull.1+"foo"+1´produces2now.Inpreviousversionthisproduced1`.

Attributenamesandparameters

PreviousversionsofArangoDBhadsometroublewithattributenamesthatcontainedthedotsymbol(.).SomecodepartsinAQLusedthedotsymboltosplitanattributenameintosub-components,soanattributenameda.bwasnotcompletelydistinguishablefromanattributeawithasub-attributeb.Thisinconsistentbehaviorsometimesallowed"hacks"toworksuchaspassingsub-attributesinabindparameterasfollows:

FORdocINcollection

FILTERdoc.@name==1

RETURNdoc

Ifthebindparameter@namecontainedthedotsymbol(e.g.@bind=a.b,itwasunclearwhetherthisshouldtriggersub-attributeaccess(i.e.doc.a.b)oraaccesstoanattributewithexactlythespecifiedname(i.e.doc["a.b"]).

ArangoDB3.0nowhandlesattributenamescontainingthedotsymbolproperly,andsendingabindparameter@name=a.bwillnowalwaystriggeranaccesstotheattributedoc["a.b"],notthesub-attributebofaindoc.

Forusersthatusedthe"hack"ofpassingbindparameterscontainingdotsymboltoaccesssub-attributes,ArangoDB3.0allowsspecifyingtheattributenamepartsasanarrayofstrings,e.g.@name=["a","b"],whichwillberesolvedtothesub-attributeaccessdoc.a.bwhenthequeryisexecuted.

Keywords

LIKEisnowakeywordinAQL.UsingLIKEineithercaseasanattributeorcollectionnameinAQLqueriesnowrequiresquoting.

SHORTEST_PATHisnowakeywordinAQL.UsingSHORTEST_PATHineithercaseasanattributeorcollectionnameinAQLqueriesnowrequiresquoting.

Subqueries

Queriesthatcontainsubqueriesthatcontaindata-modificationoperationssuchasINSERT,UPDATE,REPLACE,UPSERTorREMOVEwillnowrefusetoexecuteifthecollectionaffectedbythesubquery'sdata-modificationoperationisread-accessedinanouterscopeofthequery.

Forexample,thefollowingquerywillrefusetoexecuteasthecollectionmyCollectionismodifiedinthesubquerybutalsoread-accessedintheouterscope:

FORdocINmyCollection

LETchanges=(

FORwhatINmyCollection

FILTERwhat.value==1

REMOVEwhatINmyCollection

RETURNdoc

Itisstillpossibletowritetocollectionsfromwhichdataisreadinthesamequery,e.g.

FORdocINmyCollection

FILTERdoc.value==1

REMOVEdocINmyCollection

andtomodifydataindifferentcollectionviasubqueries.

Otherchanges

TheAQLoptimizerrule"merge-traversal-filter"thatalreadyexistedin3.0wasrenamedto"optimize-traversals".ThisshouldbeofnorelevancetoclientapplicationsexceptiftheyprogramaticallylookforappliedoptimizerrulesintheexplainoutofAQLqueries.

TheorderofresultscreatedbytheAQLfunctionsVALUES()andATTRIBUTES()wasneverguaranteedanditonlyhadthe"correct"orderingbyaccidentwheniteratingoverobjectsthatwerenotloadedfromthedatabase.Assomeofthefunctioninternalshavechanged,the"correct"orderingwillnotappearanymore,andstillnoresultorderisguaranteedbythesefunctionsunlessthesortparameterisspecified(fortheATTRIBUTES()function).

UpgradedV8version

TheV8enginethatisusedinsideArangoDBtoexecuteJavaScriptcodehasbeenupgradedfromversion4.3.61to5.0.71.39.Thenewversionshouldbemostlycompatibletotheoldversion,buttheremaybesubtledifferences,includingchangesoferrormessagetextsthrownbytheengine.Furthermore,someV8startupparametershavechangedtheirmeaningorhavebeenremovedinthenewversion.ThisisonlyrelevantwhenArangoDBorArangoShellarestartedwithacustomvalueforthe--javascript.v8-optionsstartupoption.

Amongothers,thefollowingV8optionschangeinthenewversionofArangoDB:

--es_staging:in2.8ithadthemeaningenableallcompletedharmonyfeatures,in3.0theoptionmeansenabletest-worthyharmonyfeatures(forinternaluseonly)

--strong_this:thisoptionwasn'tpresentin2.8.In3.0itmeansdon'tallow'this'toescapefromconstructorsanddefaultstotrue.

--harmony_regexps:thisoptionsmeansenable"harmonyregularexpressionextensions"andchangesitsdefaultvaluefromfalsetotrue

--harmony_proxies:thisoptionsmeansenable"harmonyproxies"andchangesitsdefaultvaluefromfalsetotrue

--harmony_reflect:thisoptionsmeansenable"harmonyReflectAPI"andchangesitsdefaultvaluefromfalsetotrue

--harmony_sloppy:thisoptionsmeansenable"harmonyfeaturesinsloppymode"andchangesitsdefaultvaluefromfalsetotrue

--harmony_tostring:thisoptionsmeansenable"harmonytoString"andchangesitsdefaultvaluefromfalsetotrue

--harmony_unicode_regexps:thisoptionsmeansenable"harmonyunicoderegexps"andchangesitsdefaultvaluefromfalsetotrue

--harmony_arrays,--harmony_array_includes,--harmony_computed_property_names,--harmony_arrow_functions,--harmony_rest_parameters,--harmony_classes,--harmony_object_literals,--harmony_numeric_literals,--harmony_unicode:theseoptionhavebeenremovedinV85.

AsaconsequenceoftheupgradetoV8version5,theimplementationoftheJavaScriptBufferobjecthadtobechanged.JavaScriptBufferobjectsinArangoDBnowalwaysstoretheirdataontheheap.ThereisnosharedpoolforsmallBuffervalues,andnopointingintoexistingBufferdatawhenextractingslices.ThischangemayincreasethecostofcreatingBufferswithshortcontentsorwhenpeekingintoexistingBuffers,butwasrequiredforsafermemorymanagementandtopreventleaks.

JavaScriptAPIchanges

ThefollowingincompatiblechangeshavebeenmadetotheJavaScriptAPIinArangoDB3.0:

TheFoxxframeworkhasbeencompletelyrewrittenfor3.0withanew,simplerandmorefamiliarAPI.TomakeFoxxservicesdevelopedfor2.8orearlierArangoDBversionsrunin3.0,theservice'smanifestfileneedstobeedited.

ToenablethelegacymodeforaFoxxservice,add"engines":{"arangodb":"^2.8.0"}(orsimilarversionrangesthatexclude3.0andup)totheservicemanifestfile(named"manifest.json",locatedintheservice'sbasedirectory).

Require

ModulesshippedwithArangoDBcannowberequiredusingthepattern@arangodb/<module>insteadoforg/arangodb/<module>,e.g.

varcluster=require("@arangodb/cluster");

Theoldformatcanstillbeusedforcompatibility:

varcluster=require("org/arangodb/cluster");

ArangoDBpriortoversion3.0allowedatransparentuseofCoffeeScriptsourcefileswiththerequire()function.FileswithafilenameextensionofcoffeewereautomaticallysentthroughaCoffeeScriptparserandtranspiledintoJavaScripton-the-fly.ThissupportisgonewithArangoDB3.0.TorunanyCoffeeScriptsourcefiles,theymustbeconvertedtoJavaScriptbytheclientapplication.

Responseobject

The@arangodb/requestresponseobjectnowstorestheparsedJSONresponsebodyinapropertyjsoninsteadofbodywhentherequestwasmadeusingthejsonoption.Thebodyinsteadcontainstheresponsebodyasastring.

EdgesAPI

Whencompletelyreplacinganedgeviaacollection'sreplace()functionthereplacingedgedatanowneedstocontainthe_fromand_toattributesforthenewedge.PreviousversionsofArangoDBdidnotrequiretheedgedatatocontain_fromand_toattributeswhenreplacinganedge,since_fromand_tovalueswereimmutableforexistingedges.

Forexample,thefollowingcallworkedinArangoDB2.8butwillfailin3.0:

db.edgeCollection.replace("myKey",{value:"test"});

TomakethisworkinArangoDB3.0,_fromand_toneedtobeaddedtothereplacementdata:

db.edgeCollection.replace("myKey",{_from:"myVertexCollection/1",_to:"myVertexCollection/2",value:"test"});

Notethatthisonlyaffectsthereplace()functionbutnotupdate(),whichwillonlyupdatethespecifiedattributesoftheedgeandleaveallothersintact.

Additionally,thefunctionsedges(),outEdges()andinEdges()withanarrayofedgeidswillnowmaketheedgeidsuniquebeforereturningtheconnectededges.Thisisprobablydesiredanyway,asresultswillbereturnedonlyonceperdistinctinputedgeid.However,itmaybreakclientapplicationsthatrelyontheoldbehavior.

DatabasesAPI

The_listDatabases()functionofthedbobjecthasbeenrenamedto_databases(),makingitconsistentwiththe_collections()function.Alsothe_listEndpoints()functionhasbeenrenamedto_endpoints().

CollectionAPI

Examplematching

ThecollectionfunctionbyExampleHash()andbyExampleSkiplist()havebeenremovedin3.0.Theirfunctionalityisprovidedbycollection'sbyExample()function,whichwillautomaticallyuseasuitableindexifpresent.

ThecollectionfunctionbyConditionSkiplist()hasbeenremovedin3.0.ThesamefunctionalitycanbeachievedbyissuinganAQLquerywiththetargetcondition,whichwillautomaticallyuseasuitableindexifpresent.

Revisionidhandling

Theexists()methodofacollectionnowthrowsanexceptionwhenthespecifieddocumentexistsbutitsrevisioniddoesnotmatchtherevisionidspecified.PreviousversionsofArangoDBsimplyreturnedfalseifeithernodocumentexistedwiththespecifiedkeyorwhentherevisioniddidnotmatch.Itwasthereforeimpossibletodistinguishthesetwocasesfromthereturnvaluealone.3.0correctsthis.Additionally,exists()inpreviousversionsalwaysreturnedabooleanifonlythedocumentkeywasgiven.3.0nowreturnsthedocument'smeta-data,whichincludesthedocument'scurrentrevisionid.

GiventhereisadocumentwithkeytestincollectionmyCollection,thenthebehaviorof3.0isasfollows:

/*testifdocumentexists.thisreturnedtruein2.8*/

db.myCollection.exists("test");

"_key":"test",

"_id":"myCollection/test",

"_rev":"9758059"

/*testifdocumentexists.thisreturnedtruein2.8*/

db.myCollection.exists({_key:"test"});

"_key":"test",

"_rev":"9758059"

/*testifdocumentexists.thisalsoreturnedfalsein2.8*/

db.myCollection.exists("foo");

/*testifdocumentwithagivenrevisionidexists.thisreturnedtruein2.8*/

db.myCollection.exists({_key:"test",_rev:"9758059"});

"_key":"test",

"_rev":"9758059"

/*testifdocumentwithagivenrevisionidexists.thisreturnedfalsein2.8*/

db.myCollection.exists({_key:"test",_rev:"1234"});

JavaScriptexception:ArangoError1200:conflict

Capconstraints

Thecapconstraintsfeaturehasbeenremoved.Thischangehasledtotheremovalofthecollectionoperationsfirst()andlast(),whichwereinternallybasedondatafromcapconstraints.

AscapconstraintshavebeenremovedinArangoDB3.0itisnotpossibletocreateanindexoftype"cap"withacollection'sensureIndex()function.ThededicatedfunctionensureCapConstraint()hasalsobeenremovedfromthecollectionAPI.

GraphBlueprintsJSModule

Thedeprecatedmodulegraph-blueprintshasbeendeleted.Allit'sfeaturesarecoveredbythegeneral-graphmodule.

GeneralGraphFluentAQLinterface

ThefluentinterfacehasbeenremovedfromArangoDB.It'sfeatureswerecompletelyoverlappingwith"aqb"whichcomespreinstalledaswell.PleaseswitchtoAQBinstead.

UndocumentedAPIs

TheundocumentedfunctionsBY_EXAMPLE_HASH()andBY_EXAMPLE_SKIPLIST(),BY_CONDITION_SKIPLIST,CPP_NEIGHBORSandCPP_SHORTEST_PATHhavebeenremoved.ThesefunctionswerealwayshiddenandnotintendedtobepartofthepublicJavaScriptAPIforcollections.

HTTPAPIchanges

CRUDoperations

ThefollowingincompatiblechangeshavebeenmadetotheHTTPAPIinArangoDB3.0:

General

TheHTTPinsertoperationsforsingledocumentsandedges(POST/_api/document)donotsupporttheURLparameter"createCollection"anymore.InpreviousversionsofArangoDBthisparametercouldbeusedtoautomaticallycreateacollectionuponinsertionofthefirstdocument.ItisnowrequiredthatthetargetcollectionalreadyexistswhenusingthisAPI,otherwiseitwillreturnanHTTP404error.ThesameistruefortheimportAPIatPOST/_api/import.

CollectionscanstillbecreatedeasilyviaaseparatecalltoPOST/_api/collectionasbefore.

The"location"HTTPheaderreturnedbyArangoDBwheninsertinganewdocumentoredgenowalwayscontainsthedatabasename.ThiswasalsothedefaultbehaviorinpreviousversionsofArangoDB,butitcouldbeoverriddenbyclientssendingtheHTTPheaderx-arango-version:1.4intherequest.ClientscancontinuetosendthisheadertoArangoDB3.0,buttheheaderwillnotinfluencethelocationresponseheadersproducedbyArangoDB3.0anymore.

AdditionallytheCRUDoperationsAPIsdonotreturnanattribute"error"intheresponsebodywithanattributevalueof"false"incaseanoperationsucceeded.

Revisionidhandling

Theoperationsforupdating,replacingandremovingdocumentscanoptionallychecktherevisionnumberofthedocumenttobeupdated,replacedorremovedsothecallercanensuretheoperationworksonaspecificversionofthedocumentandtherearenolostupdates.

PreviousversionsofArangoDBallowedpassingtherevisionidofthepreviousdocumenteitherintheHTTPheaderIf-MatchorintheURLparameterrev.Forexample,removingadocumentwithaspecificrevisionidcouldbeachievedasfollows:

curl-XDELETE\

"http://127.0.0.1:8529/_api/document/myCollection/myKey?rev=123"

ArangoDB3.0doesnotsupportpassingtherevisionidviathe"rev"URLparameteranymore.InsteadthepreviousrevisionidmustbepassedintheHTTPheaderIf-Match,e.g.

curl-XDELETE\

--header"If-Match:'123'"\

"http://127.0.0.1:8529/_api/document/myCollection/myKey"

TheURLparameter"policy"wasalsousableinpreviousversionsofArangoDBtocontrolrevisionhandling.Usingitwasredundanttospecifyingtheexpectedrevisionidviathe"rev"parameteror"If-Match"HTTPheaderandthereforesupportforthe"policy"parameterwasremovedin3.0.

Inordertocheckforapreviousrevisionidwhenupdating,replacingorremovingdocumentspleaseusetheIf-MatchHTTPheaderasdescribedabove.WhennorevisioncheckifrequiredtheHTTPheadercanbeomitted,andtheoperationswillworkonthecurrentrevisionofthedocument,regardlessofitsrevisionid.

AlldocumentsAPI

TheHTTPAPIforretrievingtheids,keysorURLsofalldocumentsfromacollectionwaspreviouslylocatedatGET/_api/document?collection=....ThisAPIwasmovedtoPUT/_api/simple/all-keysandisnowexecutedasanAQLquery.ThenameofthecollectionmustnowbepassedintheHTTPrequestbodyinsteadofintherequestURL.Thesameistrueforthe"type"parameter,whichcontrolsthetypeoftheresulttobecreated.

CallstothepreviousAPIcanbetranslatedasfollows:

old:GET/_api/document?collection=<collection>&type=<type>withoutHTTPrequestbody3.0:PUT/_api/simple/all-keyswithHTTPrequestbody{"collection":"<collection>","type":"id"}

TheresultformatofthisAPIhasalsochangedslightly.InpreviousversionscallstotheAPIreturnedaJSONobjectwithadocumentsattribute.AsthefunctionalityisbasedonAQLinternallyin3.0,theAPInowreturnsaJSONobjectwitharesultattribute.

EdgesAPI

CRUDoperations

TheAPIfordocumentsandedgeshavebeenunifiedinArangoDB3.0.TheCRUDoperationsfordocumentsandedgesarenowhandledbythesameendpointat/_api/document.ForCRUDoperationsthereisnodistinctionanymorebetweendocumentsandedgesAPI-wise.

ThatmeansCRUDoperationsconcerningedgesneedtobesenttotheHTTPendpoint/_api/documentinsteadof/_api/edge.Sendingrequeststo/_api/edgewillresultinanHTTP404errorin3.0.Thefollowingmethodsareavailableat/_api/documentfordocumentsandedge:

HTTPPOST:insertnewdocumentoredgeHTTPGET:fetchanexistingdocumentoredgeHTTPPUT:replaceanexistingdocumentoredgeHTTPPATCH:partiallyupdateanexistingdocumentoredgeHTTPDELETE:removeanexistingdocumentoredge

WhencompletelyreplacinganedgeviaHTTPPUTpleasenotethatthereplacingedgedatanowneedstocontainthe_fromand_toattributesfortheedge.PreviousversionsofArangoDBdidnotrequiresending_fromand_towhenreplacingedges,as_fromand_tovalueswereimmutableforexistingedges.

The_fromand_toattributesofedgesnowalsoneedtobepresentinsidetheedgesobjectssenttotheserver:

curl-XPOST\

--data'{"value":1,"_from":"myVertexCollection/1","_to":"myVertexCollection/2"}'\

"http://127.0.0.1:8529/_api/document?collection=myEdgeCollection"

PreviousversionsofArangoDBrequiredthe_fromand_toattributesofedgesbesentseparatelyinURLparameterfromandto:

curl-XPOST\

--data'{"value":1}'\

"http://127.0.0.1:8529/_api/edge?collection=e&from=myVertexCollection/1&to=myVertexCollection/2"

Queryingconnectededges

TheRESTAPIforqueryingconnectededgesatGET/_api/edges/<collection>willnowmaketheedgeidsuniquebeforereturningtheconnectededges.Thisisprobablydesiredanywayasresultswillnowbereturnedonlyonceperdistinctinputedgeid.However,itmaybreakclientapplicationsthatrelyontheoldbehavior.

GraphAPI

Somedata-modificationoperationsinthenamedgraphsAPIat/_api/gharialnowreturneitherHTTP202(Accepted)orHTTP201(Created)iftheoperationsucceeds.WhichstatuscodeisreturneddependsonthewaitForSyncattributeoftheaffectedcollection.InpreviousversionssomeoftheseoperationsreturnHTTP200regardlessofthewaitForSyncvalue.

ThedeprecatedgraphAPI/_api/graphhasbeenremoved.Allit'sfeaturescanbereplacedusing/_api/gharialandAQLinstead.

SimplequeriesAPI

TheRESTroutesPUT/_api/simple/firstand/_api/simple/lasthavebeenremovedentirely.TheseAPIswereresponsibleforreturningthefirst-insertedandlast-inserteddocumentsinacollection.Thisfeaturewasbuiltoncapconstraintsinternally,whichhavebeenremovedin3.0.

Callingoneoftheseendpointsin3.0willresultinanHTTP404error.

IndexesAPI

Itisnotsupportedin3.0tocreateanindexwithtypecap(capconstraint)in3.0asthecapconstraintsfeaturehasbeeremoved.CallingtheindexcreationendpointHTTPAPIPOST/_api/index?collection=...withanindextypecapwillthereforeresultinanHTTP400error.

LogentriesAPI

TheRESTrouteHTTPGET/_admin/logisnowaccessiblefromwithinalldatabases.InpreviousversionsofArangoDB,thisroutewasaccessiblefromwithinthe_systemdatabaseonly,andanHTTP403(Forbidden)wasthrownbytheserverforanyaccessfromwithinanotherdatabase.

FiguresAPI

TheRESTrouteHTTPGET/_api/collection/<collection>/figureswillnotreturnthefollowingresultattributesastheybecamemeaninglessin3.0:

shapefiles.countshapes.fileSizeshapes.countshapes.sizeattributes.countattributes.size

DatabasesandCollectionsAPIs

WhencreatingadatabaseviatheAPIPOST/_api/database,ArangoDBwillnowalwaysreturntheHTTPstatuscode202(created)iftheoperationsucceeds.PreviousversionsofArangoDBreturnedHTTP202aswell,butthisbehaviorwaschangablebysendinganHTTPheaderx-arango-version:1.4.Whensendingthisheader,previousversionsofArangoDBreturnedanHTTPstatuscode200(ok).ClientscanstillsendthisheadertoArangoDB3.0butthiswillnotinfluencetheHTTPstatuscodeproducedbyArangoDB.

The"location"headerproducedbyArangoDB3.0willnowalwayscontainthedatabasename.ThiswasalsothedefaultinpreviousversionsofArangoDB,butthebehaviorcouldbeoverriddenbysendingtheHTTPheaderx-arango-version:1.4.Clientscanstillsendtheheader,butthiswillnotmakethedatabasenameinthe"location"responseheaderdisappear.

TheresultformatforqueryingallcollectionsviatheAPIGET/_api/collectionhasbeenchanged.

PreviousversionsofArangoDBreturnedanobjectwithanattributenamedcollectionsandanattributenamednames.Bothcontainedallavailablecollections,butcollectionscontainedthecollectionsasanarray,andnamescontainedthecollectionsagain,containedinanobjectinwhichtheattributenameswerethecollectionnames,e.g.

"collections":[

{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},

{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},

"names":{

"test":{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},

"something":{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},

Thisresultstructurewasredundant,andthereforehasbeensimplifiedtojust

"result":[

{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},

{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},

inArangoDB3.0.

ReplicationAPIs

TheURLparameter"failOnUnknown"wasremovedfromtheRESTAPIGET/_api/replication/dump.Thisparametercontrolledwhetherdumpingorreplicatingedgesshouldfailifoneofthevertexcollectionslinkedintheedge's_fromor_toattributeswasnotpresentanymore.Inthiscasethe_fromand_tovaluescouldnotbetranslatedintomeaningfulidsanymore.

Thereweretwowaysforhandlingthis:

settingfailOnUnknowntotruecausedtheHTTPrequesttofail,leavingerrorhandlingtotheusersettingfailOnUnknowntofalsecausedtheHTTPrequesttocontinue,translatingthecollectionnamepartinthe_fromor_tovalueto_unknown.

InArangoDB3.0thisparameterisobsolete,as_fromand_toarestoredasself-containedstringvaluesallthetime,sotheycannotgetinvalidwhenreferencedcollectionsaredropped.

TheresultformatoftheAPIGET/_api/replication/logger-followhaschangedslightlyinthefollowingaspects:

documentsandedgesarereportedinthesameway.Thetypefordocumentinsertions/updatesandedgeinsertions/updatesisnowalways2300.PreviousversionsofArangoDBreturnedatypevalueof2300fordocumentsand2301foredges.recordsaboutinsertions,updatesorremovalsofdocumentsandedgesdonothavethekeyandrevattributesonthetop-levelanymore.Instead,keyandrevcanbeaccessedbypeekingintothe_keyand_revattributesofthedatasub-attributesofthechangerecord.

Thesameistrueforthecollection-specificchangesAPIGET/_api/replication/dump.

UsermanagementAPIs

TheRESTAPIendpointPOST/_api/userforaddingnewusersnowrequirestherequesttocontainaJSONobjectwithanattributenameduser,containingthenameoftheusertobecreated.PreviousversionsofArangoDBalsocheckedthisattribute,butadditionallylookedforanattributeusernameiftheuserattributedidnotexist.

UndocumentedAPIs

ThefollowingundocumentedHTTPRESTendpointshavebeenremovedfromArangoDB'sRESTAPI:

/_open/cerberusand/_system/cerberus:theseendpointswereintendedforsomeArangoDB-internalapplicationsonlyPUT/_api/simple/by-example-hash,PUT/_api/simple/by-example-skiplistandPUT/_api/simple/by-condition-skiplist:thesemethodsweredocumentedinearlyversionsofArangoDBbuthavebeenmarkedasnotintendedtobecalledbyenduserssinceArangoDBversion2.3.ThesemethodsshouldnothavebeenpartofanyArangoDBmanualsinceversion2.4./_api/structure:anolderunfinishedandunpromotedAPIfordataformatandtypechecks,supersededbyFoxxapplications.

AdministrationAPIs

/_admin/shutdownnowneedstobecalledwiththeHTTPDELETEmethod

HandlingofCORSrequests

ItcannowbecontrolledindetailforwhichoriginhostsCORS(Cross-originresourcesharing)requestswithcredentialswillbeallowed.ArangoDB3.0providesthestartupoption--http.trusted-originthatcanbeusedtospecifyoneormanyoriginsfromwhichCORSrequestsaretreatedas"trustworthy".

Theoptioncanbespecifiedmultipletimes,oncepertrustedorigin,e.g.

--http.trusted-originhttp://127.0.0.1:8529--http.trusted-originhttps://127.0.0.1:8599

ThiswillmaketheArangoDBserverrespondtoCORSrequestsfromtheseoriginswithanAccess-Control-Allow-CredentialsHTTPheaderwithavalueoftrue.WebbrowserscaninspectthisheaderandcanallowpassingArangoDBwebinterfacecredentials(ifstoredinthebrowser)totherequestingsite.ArangoDBwillnotforwardorprovideanycredentials.

SettingthisoptionisonlyrequiredifapplicationsonotherhostsneedtoaccesstheArangoDBwebinterfaceorotherHTTPRESTAPIsfromawebbrowserwiththesamecredentialsthattheuserhasenteredwhenloggingintothewebinterface.WhenawebbrowserfindstheAccess-Control-Allow-CredentialsHTTPresponseheader,itmayforwardthecredentialsenteredintothebrowserfortheArangoDBwebinterfacelogintotheothersite.

Thisisapotentialsecurityissue,sotherearenotrustedoriginsbydefault.Itmayberequiredtosetsometrustedoriginsifyou'replanningtoissueAJAXrequeststoArangoDBfromothersitesfromthebrowser,withthecredentialsenteredduringtheArangoDBinterfacelogin(i.e.singlesign-on).Ifsuchfunctionalityisnotused,theoptionshouldnotbeset.

Tospecifyatrustedorigin,specifytheoptiononcepertrustedoriginasshownabove.NotethatthetrustedoriginvaluesspecifiedinthisoptionwillbecomparedbytewisewiththeOriginHTTPheadervaluesentbyclients,andonlyexactmatcheswillpass.

ThereisalsothewildcardallforenablingCORSaccessfromalloriginsinatestordevelopmentsetup:

--http.trusted-originall

SettingthisoptionwillleadtotheArangoDBserverrespondingwithanAccess-Control-Allow-Credentials:trueHTTPheadertoallincomingCORSrequests.

Command-lineoptionsQuiteafewstartupoptionsinArangoDB2weredoublenegations(like--server.disable-authenticationfalse).InArangoDB3thesearenowexpressedaspositives(e.g.--server.authentication).AlsotheoptionsbetweentheArangoDBserveranditsclienttoolshavebeingunified.Forexample,theloggeroptionsarenowthesamefortheserverandtheclienttools.Additionallymanyoptionshavebeenmovedintomoreappropriatetopicsections.

Renamedoptions

Thefollowingoptionshavebeenavailablebefore3.0andhavechangedtheirnamein3.0:

--server.disable-authenticationwasrenamedto--server.authentication.Notethatthemeaningoftheoption--server.authenticationistheoppositeoftheprevious--server.disable-authentication.--server.disable-authentication-unix-socketswasrenamedto--server.authentication-unix-sockets.Notethatthemeaningoftheoption--server.authentication-unix-socketsistheoppositeoftheprevious--server.disable-authentication-unix-sockets.--server.authenticate-system-onlywasrenamedto--server.authentication-system-only.Themeaningoftheoptioninunchanged.--server.disable-statisticswasrenamedto--server.statistics.Notethatthemeaningoftheoption--server.statisticsistheoppositeoftheprevious--server.disable-statistics.--server.cafilewasrenamedto--ssl.cafile.Themeaningoftheoptionisunchanged.--server.keyfilewasrenamedto--ssl.keyfile.Themeaningoftheoptionisunchanged.--server.ssl-cachewasrenamedto--ssl.session-cache.Themeaningoftheoptionisunchanged.--server.ssl-cipher-listwasrenamedto--ssl.cipher-list.Themeaningoftheoptionisunchanged.--server.ssl-optionswasrenamedto--ssl.options.Themeaningoftheoptionisunchanged.--server.ssl-protocolwasrenamedto--ssl.protocol.Themeaningoftheoptionisunchanged.--server.backlog-sizewasrenamedto--tcp.backlog-size.Themeaningoftheoptionisunchanged.--server.reuse-addresswasrenamedto--tcp.reuse-address.Themeaningoftheoptionisunchanged.--server.disable-replication-applierwasrenamedto--database.replication-applier.Themeaningoftheoption--database.replication-applieristheoppositeoftheprevious--server.disable-replication-applier.--server.allow-method-overridewasrenamedto--http.allow-method-override.Themeaningoftheoptionisunchanged.--server.hide-product-headerwasrenamedto--http.hide-product-header.Themeaningoftheoptionisunchanged.--server.keep-alive-timeoutwasrenamedto--http.keep-alive-timeout.Themeaningoftheoptionisunchanged.--server.foxx-queueswasrenamedto--foxx.queues.Themeaningoftheoptionisunchanged.--server.foxx-queues-poll-intervalwasrenamedto--foxx.queues-poll-interval.Themeaningoftheoptionisunchanged.--no-serverwasrenamedto--server.rest-server.Notethatthemeaningoftheoption--server.rest-serveristheoppositeoftheprevious--no-server.--database.query-cache-modewasrenamedto--query.cache-mode.Themeaningoftheoptionisunchanged.--database.query-cache-max-resultswasrenamedto--query.cache-entries.Themeaningoftheoptionisunchanged.

--database.disable-query-trackingwasrenamedto--query.tracking.Themeaningoftheoption--query.trackingistheoppositeoftheprevious--database.disable-query-tracking.--log.ttywasrenamedto--log.foreground-tty.Themeaningoftheoptionisunchanged.--upgradehasbeenrenamedto--database.auto-upgrade.Incontrastto2.8thisoptionnowrequiresabooleanparameter.Toactuallyperformanautomaticdatabaseupgradeatstartupuse--database.auto-upgradetrue.Tonotperformit,use--database.auto-upgradefalse.--check-versionhasbeenrenamedto--database.check-version.--temp-pathhasbeenrenamedto--temp.path.

Logverbosity,topicsandoutputfiles

Loggingnowsupportslogtopics.Youcancontrolthesebyspecifyingalogtopicinfrontofalogleveloranoutput.Forexample

--log.levelstartup=trace--log.levelinfo

willlogmessagesconcerningstartupattracelevel,everythingelseatinfolevel.--log.levelcanbespecifiedmultipletimesatstartup,forasmanytopicsasneeded.

Somerelevantlogtopicsavailablein3.0are:

collector:informationabouttheWALcollector'sstatecompactor:informationaboutthecollectiondatafilecompactordatafiles:datafile-relatedoperationsmmap:informationaboutmemory-mappingoperationsperformance:someperformance-relatedinformationqueries:executedAQLqueriesreplication:replication-relatedinforequests:HTTPrequestsstartup:informationaboutserverstartupandshutdownthreads:informationaboutthreads

Thenewlogoption--log.output<definition>allowsdirectingtheglobalorper-topiclogoutputtodifferentoutputs.Theoutputdefinition""canbeoneof

"-"forstdin"+"forstderr"syslog://""syslog:///""file://"

Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputfordifferentlogtopics.Tosetupaper-topicoutputconfiguration,use--log.output<topic>=<definition>,e.g.

queries=file://queries.txt

logsallqueriestothefile"queries.txt".

Theoldoption--log.fileisstillavailablein3.0forconveniencereasons.In3.0itisashortcutforthemoregeneraloption--log.outputfile://filename.

Theoldoption--log.requests-fileisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption--log.outputrequests=file://....

Theoldoption--log.performanceisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption--log.levelperformance=trace.

Removedoptionsforlogging

Theoptions--log.content-filterand--log.source-filterhavebeenremoved.TheyhavemostbeenusedduringArangoDB'sinternaldevelopment.

Thesyslog-relatedoptions--log.applicationand--log.facilityhavebeenremoved.Theyaresupersededbythemoregeneral--log.outputoptionwhichcanalsohandlesyslogtargets.

Removedotheroptions

Theoption--server.default-api-compatibilitywaspresentinearlierversionofArangoDBtocontrolvariousaspectsoftheserverbehavior,e.g.HTTPreturncodesortheformatofHTTP"location"headers.ClientapplicationscouldsendanHTTPheader"x-arango-version"withaversionnumbertorequesttheserverbehaviorofacertainArangoDBversion.

Thisoptionwasonlyhonoredinahandfulofcases(describedabove)andwasremovedin3.0becausethechangesinserverbehaviorcontrolledbythisoptionwerechangedevenbeforeArangoDB2.0.Thisshouldhaveleftenoughtimeforclientapplicationstoadapttothenewbehavior,makingtheoptionsuperfluousin3.0.

Threadoptions

Theoptions--server.threadsand--scheduler.threadsnowhaveadefaultvalueof0.When--server.threadsissetto0onstartup,thesuitablenumberofthreadswillbedeterminedbyArangoDBbyaskingtheOSforthenumberofavailableCPUsandusingthatasabaseline.IfthenumberofCPUsislowerthan4,ArangoDBwillstillstart4dispatcherthreads.When--scheduler.threadsissetto0,thenArangoDBwillautomaticallydeterminethenumberofschedulerthreadstostart.Thiswillnormallycreate2schedulerthreads.

Iftheexactnumberofthreadsneedstobesetbytheadmin,thenitisstillpossibletoset--server.threadsand--scheduler.threadstonon-zerovalues.ArangoDBwillusethesevaluesandstartthatmanythreads(notethatsomethreadsmaybecreatedlazilysotheymaynotbepresentdirectlyafterstartup).

ThenumberofV8JavaScriptcontextstobecreated(--javascript.v8-contexts)nowhasadefaultvalueof0too,meaningthatArangoDBwillcreateasmanyV8contextsastherewillbedispatcherthreads(controlledbythe--server.threadsoption).Settingthisoptiontoanon-zerovaluewillcreateexactlyasmanyV8contextsasspecified.

Settingtheseoptionsexplicitlytonon-zerovaluesmaybebeneficialinenvironmentsthathavefewresources(processingtime,maximumthreadcount,availablememory).

AuthenticationThedefaultvaluefor--server.authenticationisnowtrueintheconfigurationfilesshippedwithArangoDB.Thismeanstheserverwillbestartedwithauthenticationenabledbydefault,requiringallclientconnectionstoprovideauthenticationdatawhenconnectingtoArangoDBAPIs.PreviousArangoDBversionsusedthesetting--server.disable-authenticationtrue,effectivelydisablingauthenticationbydefault.

Thedefaultvaluefor--server.authentication-system-onlyisnowtrueinArangoDB.ThatmeansthatFoxxapplicationsrunninginArangoDBwillbepublicaccessible(atleasttheywillnotuseArangoDB'sbuiltinauthenticationmechanism).OnlyrequeststoArangoDBAPIsatURLpathprefixes/_api/and/_adminwillrequireauthentication.Tochangethat,andusethebuiltinauthenticationmechanismforFoxxapplicationstoo,set--server.authentication-system-onlytofalse,andmakesuretohavetheoption--server.authenticationsettotrueaswell.

Thoughenablingtheauthenticationisrecommendedforproductionsetups,itmaybeoverkillinadevelopmentenvironment.Toturnoffauthentication,theoption--server.authenticationcanbesettofalseinArangoDB'sconfigurationfileoronthecommand-line.

WebAdminInterface

TheJavaScriptshellhasbeenremovedfromArangoDB'swebinterface.ThefunctionalitytheshellprovidedisstillfullyavailableintheArangoShell(arangosh)binaryshippedwithArangoDB.

ArangoShellandclienttools

TheArangoShell(arangosh)andtheotherclienttoolsbundledwithArangoDBcanonlyconnecttoanArangoDBserverofversion3.0orhigher.TheywillnotconnecttoanArangoDB2.8.ThisisbecausetheserverHTTPAPIshavechangedbetween2.8and3.0,andallclienttoolsusestheseAPIs.

InordertoconnecttoearlierversionsofArangoDBwiththeclienttools,anolderversionoftheclienttoolsneedstobekeptinstalled.

ThepreferrednameforthetemplatestringgeneratorfunctionaqlQueryisnowaqlandisautomaticallyavailableinarangosh.Elsewhere,itcanbeloadedlikeconstaql=require('@arangodb').aql.

Command-lineoptionsadded

Allclienttoolsin3.0provideanoption--server.max-packet-sizeforcontrollingthemaximumsizeofHTTPpacketstobehandledbytheclienttools.Thedefaultvalueis128MB,asinpreviousversionsofArangoDB.Incontrasttopreviousversionsinwhichthevaluewashard-coded,theoptionisnowconfigurable.ItcanbeincreasedtomaketheclienttoolshandleverylargeHTTPresultmessagessentbytheserver.

Command-lineoptionschanged

Forallclienttools,theoption--server.disable-authenticationwasrenamedto--server.authentication.Notethatthemeaningoftheoption--server.authenticationistheoppositeoftheprevious--server.disable-authentication.

Theoption--server.ssl-protocolwasrenamedto--ssl.protocol.Themeaningoftheoptionisunchanged.

Thecommand-lineoption--quietwasremovedfromallclienttoolsexceptarangoshbecauseithadnoeffectinthem.

Arangobench

Inordertomakeitspurposemoreapparenttheformerarangobclienttoolhasbeenrenamedtoarangobenchin3.0.

MiscellaneouschangesThechecksumcalculationalgorithmforthecollection.checksum()methodanditscorrespondingRESTAPIGET/_api/collection/<collection</checksumhaschangedin3.0.Checksumscalculatedin3.0willdifferfromchecksumscalculatedwith2.8orbefore.

TheArangoDBserverin3.0doesnotreadafileENDPOINTScontainingalistofadditionalendpointsonstartup.In2.8thisfilewasautomaticallyreadifpresentinthedatabasedirectory.

Thenamesofthesub-threadsstartedbyArangoDBhavechangedin3.0.ThisisrelevantonLinuxonly,wherethreadscanbenamedandthreadnamesmaybevisibletosystemtoolssuchastopormonitoringsolutions.

FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.8.ArangoDB2.8alsocontainsseveralbugfixesthatarenotlistedhere.Foralistofbugfixes,pleaseconsulttheCHANGELOG.

AQLimprovements

AQLGraphTraversals/PatternMatching

AQLoffersanewfeaturetotraverseoveragraphwithoutwritingJavaScriptfunctionsbutwithalltheotherfeaturesyouknowfromAQL.Forthispurpose,aspecialversionofFORvariableNameINexpressionhasbeenintroduced.

Thisspecialversionhasthefollowingformat:FORvertex-variable,edge-variable,path-variableINtraversal-expression,wheretraversal-expressionhasthefollowingformat:[depth]directionstart-vertexgraph-definitionwiththefollowinginputparameters:

depth(optional):defineshowmanystepsareexecuted.Thevaluecaneitherbeanintegervalue(e.g.3)orarangeofintegervalues(e.g.1..5).Thedefaultis1.direction:defineswhichedgedirectionsarefollowed.CanbeeitherOUTBOUND,INBOUNDorANY.start-vertex:defineswherethetraversalisstarted.Mustbean_idvalueoradocument.graph-definition:defineswhichedgecollectionsareusedforthetraversal.MustbeeitherGRAPHgraph-nameforgraphscreatedwiththegraph-module,oralistofedgecollectionsedge-col1,edge-col2,..edge-colN.

Thethreeoutputvariableshavethefollowingsemantics:

vertex-variable:Thelastvisitedvertex.edge-variable:Thelastvisitededge(optional).path-variable:Thecompletepathfromstart-vertextovertex-variable(optional).

ThetraversalstatementcanbeusedinthesamewayastheoriginalFORvariableNameINexpression,andcanbecombinedwithfiltersandotherAQLconstructs.

AsanexampleonecannowfindthefriendsofafriendforacertainuserwiththisAQLstatement:

FORfoaf,e,pathIN2ANY@startUserGRAPH"relations"

FILTERpath.edges[0].type=="friend"

FILTERpath.edges[1].type=="friend"

FILTERfoaf._id!=@startUser

RETURNDISTINCTfoaf

Optimizerruleshavebeenimplementedtogainperformanceofthetraversalstatement.Theserulesmovefilterstatementsintothetraversalstatements.t.pathswhichcanneverpassthefilterarenotemittedtothevariables.

Asanexampletakethequeryaboveandassumethereareedgesthatdonothavetype=="friend".Ifinthefirstedgestepthereissuchanon-friendedgethesecondstepswillneverbecomputedfortheseedgesastheycannotfulfillthefiltercondition.

ArrayIndexes

Hashindexesandskiplistindexescannowoptionallybedefinedforarrayvaluessothattheyindexindividualarraymembersinsteadoftheentirearrayvalue.

Todefineanindexforarrayvalues,theattributenameisextendedwiththeexpansionoperator[*]intheindexdefinition.

Example:

db._create("posts");

db.posts.ensureHashIndex("tags[*]");

Whengiventhefollowingdocument

WhatsNewin2.8

"tags":[

"AQL",

"ArangoDB",

"Index"

thisindexwillnowcontaintheindividualvalues"AQL","ArangoDB"and"Index".

Nowtheindexcanbeusedforfindingalldocumentshaving"ArangoDB"somewhereintheirtagsarrayusingthefollowingAQLquery:

FORdocINposts

FILTER"ArangoDB"INdoc.tags[*]

RETURNdoc

Itisalsopossibletocreateanindexonsub-attributesofarrayvalues.Thismakessensewhentheindexattributeisanarrayofobjects,e.g.

db._drop("posts");

db._create("posts");

db.posts.ensureIndex({type:"hash",fields:["tags[*].name"]});

db.posts.insert({tags:[{name:"AQL"},{name:"ArangoDB"},{name:"Index"}]});

db.posts.insert({tags:[{name:"AQL"},{name:"2.8"}]});

Thefollowingquerywillthenusethearrayindex:

FORdocINposts

FILTER'AQL'INdoc.tags[*].name

RETURNdoc

Arrayvalueswillautomaticallybede-duplicatedbeforebeinginsertedintoanarrayindex.

PleasenotethatfilteringusingarrayindexesonlyworksfromwithinAQLqueriesandonlyifthequeryfiltersontheindexedattributeusingtheINoperator.Theothercomparisonoperators(==,!=,>,>=,<,<=)currentlydonotusearrayindexes.

TheAQLqueryoptimizercannowuseindexesifmultiplefilterconditionsonattributesofthesamecollectionarecombinedwithlogicalORs,andiftheusageofindexeswouldcompletelycovertheseconditions.

Forexample,thefollowingqueriescannowusetwoindependentindexesonvalue1andvalue2(thelatterqueryrequiresthattheindexesareskiplistindexesduetousageofthe<and>comparisonoperators):

FORdocINcollectionFILTERdoc.value1<42||doc.value2>23RETURNdoc

Thenewoptimizerrule"sort-in-values"cannowpre-sorttheright-handsideoperandofINandNOTINoperatorssotheoperationcanuseabinarysearchwithlogarithmiccomplexityinsteadofalinearsearch.Therulewillbeappliedwhentheright-handsideoperandofanINorNOTINoperatorinafilterconditionisavariablethatisdefinedinadifferentloop/scopethantheoperatoritself.Additionally,thefilterconditionmustconsistofsolelytheINorNOTINoperationinordertoavoidanyside-effects.

Therulewillkickinforaqueriessuchasthefollowing:

LETvalues=/*someruntimeexpressionhere*/

FORdocINcollection

FILTERdoc.valueINvalues

RETURNdoc

Itwillnotbeappliedforthefollowigqueries,becausetheright-handsideoperandoftheINiseithernotavariable,orbecausetheFILTERconditionmayhavesideeffects:

WhatsNewin2.8

FORdocINcollection

FILTERdoc.valueIN/*someruntimeexpressionhere*/

RETURNdoc

LETvalues=/*someruntimeexpressionhere*/

FORdocINcollection

FILTERFUNCTION(doc.values)==23&&doc.valueINvalues

RETURNdoc

AQLfunctionsadded

POW(base,exponent):returnsthebasetotheexponentexp

UNSET_RECURSIVE(document,attributename,...):recursivelyremovestheattributesattributename(canbeoneormany)fromdocumentanditssub-documents.Allotherattributeswillbepreserved.Multipleattributenamescanbespecifiedbyeitherpassingmultipleindividualstringargumentnames,orbypassinganarrayofattributenames:

UNSET_RECURSIVE(doc,'_id','_key','foo','bar')

UNSET_RECURSIVE(doc,['_id','_key','foo','bar'])

IS_DATESTRING(value):returnstrueifvalueisastringthatcanbeusedinadatefunction.Thisincludespartialdatessuchas2015or2015-10andstringscontaininginvaliddatessuchas2015-02-31.Thefunctionwillreturnfalseforallnon-stringvalues,evenifsomeofthemmaybeusableindatefunctions.

theArangoShellnowprovidestheconveniencefunctiondb._explain(query)forretrievingahuman-readableexplanationofAQLqueries.Thisfunctionisashorthandforrequire("org/arangodb/aql/explainer").explain(query).

theAQLqueryoptimizernowautomaticallyconvertsLENGTH(collection-name)toanoptimizedexpressionthatreturnsthenumberofdocumentsinacollection.PreviousversionsofArangoDBreturnedawarningwhenusingthisexpressionandalsoenumeratedalldocumentsinthecollection,whichwasinefficient.

improvedperformanceofskippingovermanydocumentsinanAQLquerywhennoindexesandnofiltersareused,e.g.

FORdocINcollection

LIMIT1000000,10

RETURNdoc

addedclusterexecutionsiteinfoinexecutionplanexplainoutputforAQLqueries

for30+AQLfunctionsthereisnowanadditionalimplementationinC++thatremovestheneedforinternaldataconversionwhenthefunctioniscalled

theAQLeditorinthewebinterfacenowsupportsusingbindparameters

Deadlockdetection

ArangoDB2.8nowhasanautomaticdeadlockdetectionfortransactions.

Adeadlockisasituationinwhichtwoormoreconcurrentoperations(usertransactionsorAQLqueries)trytoaccessthesameresources(collections,documents)andneedtowaitfortheotherstofinish,butnoneofthemcanmakeanyprogress.

Incaseofsuchadeadlock,therewouldbenoprogressforanyoftheinvolvedtransactions,andnoneoftheinvolvedtransactionscouldevercomplete.Thisiscompletelyundesirable,sothenewautomaticdeadlockdetectionmechanisminArangoDBwillautomaticallykickinandabortoneofthetransactionsinvolvedinsuchadeadlock.Abortingmeansthatallchangesdonebythetransactionwillberolledbackanderror29(deadlockdetected)willbethrown.

WhatsNewin2.8

Clientcode(AQLqueries,usertransactions)thataccessesmorethanonecollectionshouldbeawareofthepotentialofdeadlocksandshouldhandletheerror29(deadlockdetected)properly,eitherbypassingtheexceptiontothecallerorretryingtheoperation.

Replication

Thefollowingimprovementsforreplicationhavebeenmadein2.8(note:mostofthemhavebeenbackportedtoArangoDB2.7aswell):

addedautoResyncconfigurationparameterforcontinuousreplication.

Whensettotrue,areplicationslavewillautomaticallytriggerafulldatare-synchronizationwiththemasterwhenthemastercannotprovidethelogdatatheslavehadaskedfor.NotethatautoResyncwillonlyworkwhentheoptionrequireFromPresentisalsosettotrueforthecontinuousreplication,orwhenthecontinuoussyncerisstartedanddetectsthatnostarttickispresent.

Automaticre-synchronizationmaytransferalotofdatafromthemastertotheslaveandmaybeexpensive.Itisthereforeturnedoffbydefault.Whenturnedoff,theslavewillneverperformanautomaticre-synchronizationwiththemaster.

addedidleMinWaitTimeandidleMaxWaitTimeconfigurationparametersforcontinuousreplication.

Theseparameterscanbeusedtocontroltheminimumandmaximumwaittimetheslavewill(intentionally)idleandnotpollformasterlogchangesincasethemasterhadsentthefulllogsalready.TheidleMaxWaitTimevaluewillonlybeusedwhenadapativePollingissettotrue.WhenadaptivePollingisdisabled,onlyidleMinWaitTimewillbeusedasaconstanttimespaninwhichtheslavewillnotpollthemasterforfurtherchanges.Thedefaultvaluesare0.5secondsforidleMinWaitTimeand2.5secondsforidleMaxWaitTime,whichcorrespondtothehard-codedvaluesusedinpreviousversionsofArangoDB.

addedinitialSyncMaxWaitTimeconfigurationparameterforinitialandcontinuousreplication

Thisoptioncontrolsthemaximumwaittime(inseconds)thattheinitialsynchronizationwillwaitforaresponsefromthemasterwhenfetchinginitialcollectiondata.Ifnoresponseisreceivedwithinthistimeperiod,theinitialsynchronizationwillgiveupandfail.ThisoptionisalsorelevantforcontinuousreplicationincaseautoResyncissettotrue,asthenthecontinuousreplicationmaytriggerafulldatare-synchronizationincasethemastercannotthelogdatatheslavehadaskedfor.

HTTPrequestssentfromtheslavetothemasterduringinitialsynchronizationwillnowberetriediftheyfailwithconnectionproblems.

theinitialsynchronizationnowlogsitsprogresssoitcanbequeriedusingtheregularreplicationstatuscheckAPIs.

addedasyncattributeforsyncandsyncCollectionoperationscalledfromtheArangoShell.Setthingthisattributetotruewillmakethesynchronizationjobontheservergointothebackground,sothattheshelldoesnotblock.ThestatusofthestartedasynchronoussynchronizationjobcanbequeriedfromtheArangoShelllikethis:

/*startsinitialsynchronization*/

varreplication=require("org/arangodb/replication");

varid=replication.sync({

username:"myuser",

async:true

/*nowquerytheidofthereturnedasyncjobandprintthestatus*/

TheresultofgetSyncResult()willbefalsewhiletheserver-sidejobhasnotcompleted,anddifferenttofalseifithascompleted.Whenithascompleted,alljobresultdetailswillbereturnedbythecalltogetSyncResult().

thewebadmininterfacedashboardnowshowsaserver'sreplicationstatusatthebottomofthepage

WebAdminInterface

Thefollowingimprovementshavebeenmadeforthewebadmininterface:

theAQLeditornowhassupportforbindparameters.Thebindparametervaluescanbeeditedinthewebinterfaceandsavedwithaqueryforfutureuse.

WhatsNewin2.8

theAQLeditornowallowscancelingrunningqueries.Thiscanbeusedtocancellong-runningquerieswithoutswitchingtothequerymanagementsection.

thedashboardnowprovidesinformationabouttheserver'sreplicationstatusatthebottomofthepage.Thiscanbeusedtotrackeitherthestatusofaone-timesynchronizationorthecontinuousreplication.

thecompactionstatusandsomestatusinternalsaboutcollectionsarenowdisplayedinthedetailviewforacollectioninthewebinterface.Thesedatacanbeusedfordebuggingcompactionissues.

unloadingacollectionviathewebinterfacewillnowtriggergarbagecollectioninallv8contextsandforceaWALflush.Thisincreasesthechancesofperfomingtheunloadfaster.

thestatusterminologyforcollectionsforwhichanunloadrequesthasbeenissuedviathewebinterfacewaschangedfromintheprocessofbeingunloadedtowillbeunloaded.Thisismoreaccurateastheactualunloadmaybepostponeduntillateriftherearestillreferencespointingtodatainthecollection.

Foxximprovementsthemoduleresolutionusedbyrequirenowbehavesmorelikeinnode.js

theorg/arangodb/requestmodulenowreturnsresponsebodiesforerrorresponsesbydefault.Theoldbehaviorofnotreturningbodiesforerrorresponsescanbere-enabledbyexplicitlysettingtheoptionreturnBodyOnErrortofalse

Miscellaneouschanges

Thestartupoption--server.hide-product-headercanbeusedtomaketheservernotsendtheHTTPresponseheader"Server:ArangoDB"initsHTTPresponses.ThiscanbeusedtoconcealtheservermakefromHTTPclients.Bydefault,theoptionisturnedoffsotheheaderisstillsentasusual.

arangodumpandarangorestorenowhavebettererrorreporting.Additionally,arangodumpwillnowfailbydefaultwhentryingtodumpedgesthatrefertoalreadydroppedcollections.Thiscanbecircumventedbyspecifyingtheoption--forcetruewheninvokingarangodump.

arangoimpnowprovidesanoption--create-collection-typetospecifythetypeofthecollectiontobecreatedwhen--create-collectionissettotrue.Previously--create-collectionalwayscreateddocumentcollectionsandthecreationofedgecollectionswasnotpossible.

WhatsNewin2.8

Keywordsadded

ThefollowingAQLkeywordswereaddedinArangoDB2.8:

OUTBOUND

INBOUND

AGGREGATE

Usageofthesekeywordsforcollectionnames,variablenamesorattributenamesinAQLquerieswillnotbepossiblewithoutquoting.Forexample,thefollowingAQLquerywillstillworkasitusesaquotedcollectionnameandaquotedattributename:

FORdocIN`OUTBOUND`

RETURNdoc.`any`

Changedbehavior

TheAQLfunctionsNEARandWITHINnowhavestrictervalidationsfortheirinputparameterslimit,radiusanddistance.Theymaynowthrowexceptionswheninvalidparametersarepassedthatmayhavenotledtoexceptionsinpreviousversions.

Additionally,theexpansion([*])operatorinAQLhaschangeditsbehaviorwhenhandlingnon-arrayvalues:

InArangoDB2.8,callingtheexpansionoperatoronanon-arrayvaluewillalwaysreturnanemptyarray.PreviousversionsofArangoDBexpandednon-arrayvaluesbycallingtheTO_ARRAY()functionforthevalue,whichforexamplereturnedanarraywithasinglevalueforboolean,numericandstringinputvalues,andanarraywiththeobject'svaluesforanobjectinputvalue.Thisbehaviorwasinconsistentwithhowtheexpansionoperatorworksforthearrayindexesin2.8,sothebehaviorisnowunified:

iftheleft-handsideoperandof[*]isanarray,thearraywillbereturnedasiswhencalling[*]onitiftheleft-handsideoperandof[*]isnotanarray,anemptyarraywillbereturnedby[*]

AQLqueriesthatrelyontheoldbehaviorcanbechangedbyeithercallingTO_ARRAYexplicitlyorbyusingthe[*]atthecorrectposition.

Thefollowingexamplequerywillchangeitsresultin2.8comparedto2.7:

LETvalues="foo"RETURNvalues[*]

In2.7thequeryhasreturnedthearray["foo"],butin2.8itwillreturnanemptyarray[].Tomakeitreturnthearray["foo"]again,anexplicitTO_ARRAYfunctioncallisneededin2.8(whichinthiscaseallowstheremovalofthe[*]operatoraltogether).Thisalsoworksin2.7:

LETvalues="foo"RETURNTO_ARRAY(values)

Anotherexample:

LETvalues=[{name:"foo"},{name:"bar"}]

RETURNvalues[*].name[*]

Theabovereturned[["foo"],["bar"]]in2.7.In2.8itwillreturn[[],[]],becausethevalueofname`isnotanarray.Tochangetheresultstothe2.7style,thequerycanbechangedto

LETvalues=[{name:"foo"},{name:"bar"}]

RETURNvalues[*RETURNTO_ARRAY(CURRENT.name)]

Theabovealsoworksin2.7.Thefollowingtypesofquerieswon'tchange:

LETvalues=[1,2,3]RETURNvalues[*]

LETvalues=[{name:"foo"},{name:"bar"}]RETURNvalues[*].name

LETvalues=[{names:["foo","bar"]},{names:["baz"]}]RETURNvalues[*].names[*]

LETvalues=[{names:["foo","bar"]},{names:["baz"]}]RETURNvalues[*].names[**]

Deadlockhandling

Clientapplicationsshouldbepreparedtohandleerror29(deadlockdetected)thatArangoDBmaynowthrowwhenitdetectsadeadlockacrossmultipletransactions.Whenaclientapplicationreceiveserror29,itshouldretrytheoperationthatfailed.

TheerrorcanonlyoccurforAQLqueriesorusertransactionsthatinvolvemorethanasinglecollection.

Optimizer

TheAQLexecutionnodetypeIndexRangeNodewasreplacedwithanewmorecapableexecutionnodetypeIndexNode.ThatmeansinexecutionplanexplainoutputtherewillbenomoreIndexRangeNodesbutonlyIndexNode.Thisaffectsexplainoutputthatcanberetrievedviarequire("org/arangodb/aql/explainer").explain(query),db._explain(query),andtheHTTPqueryexplainAPI.

TheoptimizerrulethatmakesAQLqueriesactuallyuseindexeswasalsorenamedfromuse-index-rangetouse-indexes.Againthisaffectsexplainoutputthatcanberetrievedviarequire("org/arangodb/aql/explainer").explain(query),db._explain(query),andtheHTTPqueryexplainAPI.

Thequeryoptimizerruleremove-collect-intowasrenamedtoremove-collect-variables.Thisaffectsexplainoutputthatcanberetrievedviarequire("org/arangodb/aql/explainer").explain(query),db._explain(query),andtheHTTPqueryexplainAPI.

HTTPAPI

Whenaserver-sideoperationgotcanceledduetoanexplicitclientcancelrequestviaHTTPDELETE/_api/job,previousversionsofArangoDBreturnedanHTTPstatuscodeof408(requesttimeout)fortheresponseofthecanceledoperation.

TheHTTPreturncode408hascausedproblemswithsomeclientapplications.Somebrowsers(e.g.Chrome)handleda408responsebyresendingtheoriginalrequest,whichistheoppositeofwhatisdesiredwhenajobshouldbecanceled.

ThereforeArangoDBwillreturnHTTPstatuscode410(gone)forcanceledoperationsfromversion2.8on.

ModelandRepository

DuetocompatibilityissuestheModelandRepositorytypesarenolongerimplementedasES2015classes.

Thepre-2.7"extend"stylesubclassingissupportedagainandwillnotemitanydeprecationwarnings.

varMyModel=Foxx.Model.extend({

schema:{/*...*/}

Moduleresolution

ThebehavioroftheJavaScriptmoduleresolutionusedbytherequirefunctionhasbeenmodifiedtoimprovecompatibilitywithmoduleswrittenforNode.js.

Specifically

absolutepaths(e.g./some/absolute/path)arenowalwaysinterpretedasabsolutefilesystempaths,relativetothefilesystemroot

globalnames(e.g.global/name)arenowfirstintepretedasreferencestomodulesresidinginarelevantnode_modulesfolder,abuilt-inmoduleoramatchingdocumentintheinternal_modulescollection,andonlyresolvedtolocalfilepathsifnoothermatchisfound

Previouslythetwoformatsweretreatedinterchangeablyandwouldberesolvedtolocalfilepathsfirst,leadingtoproblemswhenlocalfilesusedthesamenamesasothermodules(e.g.alocalfilechai.jswouldcauseproblemswhentryingtoloadthechaimoduleinstalledinnode_modules).

Formoreinformationseetheblogannouncementofthischangeandtheupgradeguide.

Moduleorg/arangodb/request

Themodulenowalwaysreturnsresponsebodies,evenforerrorresponses.Inversionspriorto2.8themodulewouldsilentlydropresponsebodiesiftheresponseheaderindicatedanerror.

TheoldbehaviorofnotreturningbodiesforerrorresponsescanberestoredbyexplicitlysettingtheoptionreturnBodyOnErrortofalse:

letresponse=request({

returnBodyOnError:false

Garbagecollection

TheV8garbagecollectionstrategywasslightlyadjustedsothatiteventuallyhappensinallV8contextsthatholdV8externalobjects(referencestoArangoDBdocumentsandcollections).Thisenablesabettercleanupoftheseresourcesandpreventsotherprocessessuchascompactionbeingstalledwhilewaitingfortheseresourcestobereleased.

InthiscontextthedefaultvaluefortheJavaScriptgarbagecollectionfrequency(--javascript.gc-frequency)wasalsoincreasedfrom10secondsto15seconds,aslessinternaloperationsinArangoDBarecarriedoutinJavaScript.

Clienttools

arangodumpwillnowfailbydefaultwhentryingtodumpedgesthatrefertoalreadydroppedcollections.Thiscanbecircumventedbyspecifyingtheoption--forcetruewheninvokingarangodump

Performanceimprovements

Indexbuckets

Theprimaryindexesandhashindexesofcollectionscannowbesplitintomultipleindexbuckets.ThisoptionwasavailableforedgeindexesonlyinArangoDB2.6.

Abucketcanbeconsideredacontainerforaspecificrangeofindexvalues.Forprimary,hashandedgeindexes,determiningtheresponsiblebucketforanindexvalueisdonebyhashingtheactualindexvalueandapplyingasimplearithmeticoperationonthehash.

Becauseanindexvaluewillbepresentinatmostonebucketandbucketsareindependent,usingmultiplebucketsprovidesthefollowingbenefits:

initiallybuildingthein-memoryindexdatacanbeparallelizedevenforasingleindex,withonethreadperbucket(orwiththreadsbeingresponsibleformorethanonebucketatatime).Thiscanhelpreducingtheloadingtimeforcollections.

resizinganindexwhenitisabouttorunoutofreservespaceisperformedperbucket.Aseachbucketonlycontainsafractionoftheentireindex,resizingandrehashingabucketismuchfasterandlessintrusivethanresizingandrehashingtheentireindex.

Whencreatingnewcollections,thedefaultnumberofindexbucketsis8sinceArangoDB2.7.Inpreviousversions,thedefaultvaluewas1.Thenumberofbucketscanalsobeadjustedforexistingcollectionssotheycanbenefitfromtheoptimizations.Thenumberofindexbucketscanbesetforacollectionatanytimebyusingacollection'spropertiesfunction:

db.collection.properties({indexBuckets:16});

Thenumberofindexbucketsmustbeapowerof2.

Pleasenotethatforbuildingtheindexdataformultiplebucketsinparallelitisrequiredthatacollectioncontainsasignificantamountofdocumentsbecauseforalownumberofdocumentstheoverheadofparallelizationwilloutweighitsbenefits.Thecurrentthresholdvalueis256kdocuments,butthisvaluemaychangeinfutureversionsofArangoDB.Additionally,theconfigurationoption--database.index-threadswilldeterminehowmanyparallelthreadsmaybeusedforbuildingtheindexdata.

Fasterupdateandremoveoperationsinnon-uniquehashindexes

TheuniquehashindexesinArangoDBprovidedanamortizedO(1)lookup,insert,updateandremoveperformance.Non-uniquehashindexesprovidedamortizedO(1)insertperformance,buthadworseperformanceforupdateandremoveoperationsfornon-uniquevalues.Fordocumentswiththesameindexvalue,theymaintainedalistofcollisions.Whenadocumentwasupdatedorremoved,thatexactdocumenthadtobefoundinthecollisionslistfortheindexvalue.WhilegettingtothestartofthecollisionslistwasO(1),scanningthelisthadO(n)performanceintheworstcase(withnbeingthenumberofdocumentswiththesameindexvalue).Overall,thismadeupdateandremoveoperationsinnon-uniquehashindexesslowiftheindexcontainedmanyduplicatevalues.

ThishasbeenchangedinArangoDB2.7sothatnon-uniquehashindexesnowalsoprovideupdateandremoveoperationswithanamortizedcomplexityofO(1),eveniftherearemanyduplicates.

Resizingnon-uniquehashindexesnowalsodoesn'trequirelookingintothedocumentdata(whichmayinvolveadiskaccess)becausetheindexmaintainssomeinternalcachevalueperdocument.Whenresizingandrehashingtheindex(oranindexbucket),theindexwillfirstcompareonlythecachevaluesbeforepeekingintotheactualdocuments.Thischangecanalsoleadtoreducedindexresizingtimes.

Throughputenhancements

TheArangoDB-internalimplementationsfordispatchingrequests,keepingstatisticsandassigningV8contextstothreadshavebeenimprovedinordertouselesslocks.Thesechangesallowhigherconcurrencyandthroughputinthesecomponents,whichcanalsomaketheserverhandlemorerequestsinagivenperiodoftime.

WhatsNewin2.7

Whatgainscanbeexpecteddependsonwhichoperationsareexecuted,buttherearereal-worldcasesinwhichthroughputincreasedbybetween25%and70%whencomparedto2.6.

Madvisehints

TheLinuxvariantforArangoDBprovidestheOSwithmadvisehintsaboutindexmemoryanddatafilememory.Thesehintscanspeedupthingswhenmemoryistight,inparticularatcollectionloadtimebutalsoforrandomaccesseslater.ThereisnoformalguaranteethattheOSactuallyusesthemadvisehintsprovidedbyArangoDB,butactualmeasurementshaveshownimprovementsforloadingbiggercollections.

AQLimprovements

Additionaldatefunctions

ArangoDB2.7providesseveralextraAQLfunctionsfordateandtimecalculationandmanipulation.ThesefunctionswerecontributedbyGitHubusers@CoDEmanXand@friday.Abigthanksfortheirwork!

Thefollowingextradatefunctionsareavailablefrom2.7on:

DATE_DAYOFYEAR(date):Returnsthedayofyearnumberofdate.Thereturnvaluesrangefrom1to365,or366inaleapyearrespectively.

DATE_ISOWEEK(date):ReturnstheISOweekdateofdate.Thereturnvaluesrangefrom1to53.Mondayisconsideredthefirstdayoftheweek.Therearenofractionalweeks,thusthelastdaysinDecembermaybelongtothefirstweekofthenextyear,andthefirstdaysinJanuarymaybepartofthepreviousyear'slastweek.

DATE_LEAPYEAR(date):Returnswhethertheyearofdateisaleapyear.

DATE_QUARTER(date):Returnsthequarterofthegivendate(1-based):

1:January,February,March2:April,May,June3:July,August,September4:October,November,December

DATE_DAYS_IN_MONTH(date):Returnsthenumberofdaysindate'smonth(28..31).

DATE_ADD(date,amount,unit):Addsamountgiveninunittodateandreturnsthecalculateddate.

unitcanbeeitherofthefollowingtospecifythetimeunittoaddorsubtract(case-insensitive):

y,year,yearsm,month,monthsw,week,weeksd,day,daysh,hour,hoursi,minute,minutess,second,secondsf,millisecond,milliseconds

amountisthenumberofunitstoadd(positivevalue)orsubtract(negativevalue).

DATE_SUBTRACT(date,amount,unit):Subtractsamountgiveninunitfromdateandreturnsthecalculateddate.

ItworksthesameasDATE_ADD(),exceptthatitsubtracts.ItisequivalenttocallingDATE_ADD()withanegativeamount,exceptthatDATE_SUBTRACT()canalsosubtractISOdurations.NotethatnegativeISOdurationsarenotsupported(i.e.startingwith-P,like-P1Y).

DATE_DIFF(date1,date2,unit,asFloat):Calculatethedifferencebetweentwodatesingiventimeunit,optionallywithdecimalplaces.Returnsanegativevalueifdate1isgreaterthandate2.

DATE_COMPARE(date1,date2,unitRangeStart,unitRangeEnd):Comparetwopartialdatesandreturntrueiftheymatch,falseotherwise.Thepartstocomparearedefinedbyarangeoftimeunits.

WhatsNewin2.7

Thefullrangeis:years,months,days,hours,minutes,seconds,milliseconds.PasstheunittostartfromasunitRangeStart,andtheunittoendwithasunitRangeEnd.Allunitsinbetweenwillbecompared.LeaveoutunitRangeEndtoonlycompareunitRangeStart.

DATE_FORMAT(date,format):Formatadateaccordingtothegivenformatstring.Itsupportsthefollowingplaceholders(case-insensitive):

%t:timestamp,inmillisecondssincemidnight1970-01-01%z:ISOdate(0000-00-00T00:00:00.000Z)%w:dayofweek(0..6)%y:year(0..9999)%yy:year(00..99),abbreviated(lasttwodigits)%yyyy:year(0000..9999),paddedtolengthof4%yyyyyy:year(-009999..+009999),withsignprefixandpaddedtolengthof6%m:month(1..12)%mm:month(01..12),paddedtolengthof2%d:day(1..31)%dd:day(01..31),paddedtolengthof2%h:hour(0..23)%hh:hour(00..23),paddedtolengthof2%i:minute(0..59)%ii:minute(00..59),paddedtolengthof2%s:second(0..59)%ss:second(00..59),paddedtolengthof2%f:millisecond(0..999)%fff:millisecond(000..999),paddedtolengthof3%x:dayofyear(1..366)%xxx:dayofyear(001..366),paddedtolengthof3%k:ISOweekdate(1..53)%kk:ISOweekdate(01..53),paddedtolengthof2%l:leapyear(0or1)%q:quarter(1..4)%a:daysinmonth(28..31)%mmm:abbreviatedEnglishnameofmonth(Jan..Dec)%mmmm:Englishnameofmonth(January..December)%www:abbreviatedEnglishnameofweekday(Sun..Sat)%wwww:Englishnameofweekday(Sunday..Saturday)%&:specialescapesequenceforrareoccasions%%:literal%%:ignored

RETURNDISTINCT

Toreturnuniquevaluesfromaquery,AQLnowprovidestheDISTINCTkeyword.ItcanbeusedasamodifierforRETURNstatements,asashorteralternativetothealreadyexistingCOLLECTstatement.

Forexample,thefollowingqueryonlyreturnsdistinct(unique)statusattributevaluesfromthecollection:

FORdocINcollection

RETURNDISTINCTdoc.status

RETURNDISTINCTisnotallowedonthetop-levelofaqueryifthereisnoFORloopinfrontofit.RETURNDISTINCTisallowedinsubqueries.

RETURNDISTINCTensuresthatthevaluesreturnedaredistinct(unique),butdoesnotguaranteeanyorderofresults.Inordertohavecertainresultorder,anadditionalSORTstatementmustbeaddedtoaquery.

Shorthandobjectnotation

WhatsNewin2.7

AQLnowprovidesashorthandnotationforobjectliteralsinthestyleofES6objectliterals:

LETname="Peter"

LETage=42

RETURN{name,age}

Thisisequivalenttothepreviouslyavailablecanonicalform,whichisstillavailableandsupported:

LETname="Peter"

LETage=42

RETURN{name:name,age:age}

Arrayexpansionimprovements

Thealreadyexisting[*]operatorhasbeenimprovedwithoptionalfilteringandprojectionandlimitcapabilities.

Forexample,considerthefollowingexamplequerythatfiltersvaluesfromanarrayattribute:

FORuINusers

RETURN{

name:u.name,

friends:(

FORfINu.friends

FILTERf.age>u.age

RETURNf.name

Withthe[*]operator,thisquerycanbesimplifiedto

FORuINusers

RETURN{name:u.name,friends:u.friends[*FILTERCURRENT.age>u.age].name}

Thepseudo-variableCURRENTcanbeusedtoaccessthecurrentarrayelement.TheFILTERconditioncanrefertoCURRENToranyvariablesvalidintheouterscope.

Toreturnaprojectionofthecurrentelement,therecannowbeaninlineRETURN:

FORuINusers

RETURNu.friends[*RETURNCONCAT(CURRENT.name,"isafriendof",u.name)]

whichisthesimplifiedvariantfor:

FORuINusers

RETURN(

FORfriendINu.friends

RETURNCONCAT(friend.name,"isafriendof",u.name)

Arraycontraction

Inordertocollapse(orflatten)resultsinnestedarrays,AQLnowprovidesthe[**]operator.Itworkssimilartothe[*]operator,butadditionallycollapsesnestedarrays.Howmanylevelsarecollapsedisdeterminedbytheamountof*charactersused.

Forexample,considerthefollowingquerythatproducesanestedresult:

FORuINusers

RETURNu.friends[*].name

The[**]operatorcannowbeappliedtogetridofthenestedarrayandturnitintoaflatarray.Wesimplyapplythe[**]onthepreviousqueryresult:

WhatsNewin2.7

RETURN(

FORuINusersRETURNu.friends[*].name

Templatequerystrings

AssemblingquerystringsinJavaScripthasbeenerror-pronewhenusingsimplestringconcatenation,especiallybecauseplainJavaScriptstringsdonothavemultiline-support,andbecauseofpotentialparameterinjectionissues.WhilemultilinequerystringscanbeassembledwithES6templatestringssinceArangoDB2.5,andquerybindparametersaretheresinceArangoDB1.0topreventparameterinjection,therewasnoJavaScript-ysolutiontocombinethese.

ArangoDB2.7nowprovidesanES6templatestringgeneratorfunctionthatcanbeusedtoeasilyandsafelyassembleAQLqueriesfromJavaScript.JavaScriptvariablesandexpressionscanbeusedeasilyusingregularES6templatestringsubstitutions:

letname='test';

letattributeName='_key';

letquery=aqlQuery`FORuINusers

FILTERu.name==${name}

RETURNu.${attributeName}`;

db._query(query);

ThisismorelegiblethanwhenusingaplainJavaScriptstringandalsodoesnotrequiredefiningthebindparametervaluesseparately:

letname='test';

letattributeName='_key';

letquery="FORuINusers"+

"FILTERu.name==@name"+

"RETURNu.@attributeName";

db._query(query,{

attributeName

TheaqlQuerytemplatestringgeneratorwillalsohandlecollectionobjectsautomatically:

db._query(aqlQuery`FORuIN${db.users}RETURNu.name`);

NotethatwhiletemplatestringsareavailableintheJavaScriptfunctionsprovidedtobuildqueries,theyaren'tafeatureofAQLitself.AQLcouldalwayshandlemultilinequerystringsandprovidedbindparameters(@...)forseparatingthequerystringandtheparametervalues.TheaqlQuerytemplatestringgeneratorfunctionwilltakecareofthisseparation,too,butwilldoitbehindthescenes.

AQLqueryresultcache

TheAQLqueryresultcachecanoptionallycachethecompleteresultsofallorjustselectedAQLqueries.Itcanbeoperatedinthefollowingmodes:

off:thecacheisdisabled.Noqueryresultswillbestoredon:thecachewillstoretheresultsofallAQLqueriesunlesstheircacheattributeflagissettofalsedemand:thecachewillstoretheresultsofAQLqueriesthathavetheircacheattributesettotrue,butwillignoreallothers

Themodecanbesetatserverstartupusingthe--database.query-cache-modeconfigurationoptionandlaterchangedatruntime.Thedefaultvalueisoff,meaningthatthequeryresultcacheisdisabled.Thisisbecausethecachemayconsumeadditionalmemorytokeepqueryresults,andalsobecauseitmustbeinvalidatedwhenchangeshappenincollectionsforwhichresultshavebeencached.

Thequeryresultcachemaythereforehavepositiveornegativeeffectsonqueryexecutiontimes,dependingontheworkload:itwillnotmakemuchsenseturningonthecacheinwrite-onlyorwrite-mostlyscenarios,butthecachemaybeverybeneficialincaseworkloadsareread-onlyorread-mostly,andqueryarecomplex.

Ifthequerycacheisoperatedindemandmode,itcanbecontrolledperqueryifthecacheshouldbecheckedforaresult.

WhatsNewin2.7

Optimizer

TheAQLoptimizerrulepatch-update-statementshasbeenadded.ThisrulecanoptimizecertainAQLUPDATEqueriesthatupdatedocumentsintheacollectionthattheyalsoiterateover.

Forexample,thefollowingqueryreadsdocumentsfromacollectioninordertoupdatethem:

FORdocINcollection

UPDATEdocWITH{newValue:doc.oldValue+1}INcollection

Inthiscase,onlyasinglecollectionisaffectedbythequery,andthereisnoindexlookupinvolvedtofindtheto-be-updateddocuments.Inthiscase,theUPDATEquerydoesnotrequiretakingafull,memory-intensivesnapshotofthecollection,butitcanbeperformedinsmallchunks.Thiscanleadtomemorysavingswhenexecutingsuchqueries.

Functioncallargumentsoptimization

ThisoptimizationwillleadtoargumentsinfunctioncallsinsideAQLqueriesnotbeingcopiedbutbeingpassedbyreference.ThismayspeedupcallstofunctionswithbiggerargumentvaluesorqueriesthatcallAQLfunctionsalotoftimes.

WebAdminInterface

Thewebinterfacenowhasanewdesign.

The"Applications"tabinthewebinterfaceshasbeenrenamedto"Services".

TheArangoDBAPIdocumentationhasbeenmovedfromthe"Tools"menutothe"Links"menu.ThenewdocumentationisbasedonSwagger2.0andopensinaseparatewebpage.

Foxximprovements

ES2015Classes

AllFoxxconstructorshavebeenreplacedwithES2015classesandcanbeextendedusingtheclasssyntax.TheextendmethodisstillsupportedatthemomentbutwillbecomedeprecatedinArangoDB2.8andremovedinArangoDB2.9.

Before:

varMyModel=Foxx.Model.extend({

schema:{/*...*/}

After:

classMyModelextendsFoxx.Model{

MyModel.prototype.schema={/*...*/};

Confidentialconfiguration

Itisnowpossibletospecifyconfigurationoptionswiththetypepassword.ThepasswordtypeisequivalenttothetexttypebutwillbemaskedinthewebfrontendtopreventaccidentalexposureofconfidentialoptionslikeAPIkeysandpasswordswhenconfiguringyourFoxxapplication.

WhatsNewin2.7

Dependencies

Thesyntaxforspecifyingdependenciesinmanifestshasbeenextendedtoallowspecifyingoptionaldependencies.Unmetoptionaldependencieswillnotpreventanappfrombeingmounted.Thetraditionalshorthandsyntaxforspecifyingnon-optionaldependencieswillstillbesupportedintheupcomingversionsofArangoDB.

Before:

"dependencies":{

"notReallyNeeded":"users:^1.0.0",

"totallyNecessary":"sessions:^1.0.0"

After:

"dependencies":{

"notReallyNeeded":{

"name":"users",

"version":"^1.0.0",

"required":false

"totallyNecessary":{

"name":"sessions",

"version":"^1.0.0"

Replication

TheexistingreplicationHTTPAPIhasbeenextendedwithmethodsthatreplicationclientscanusetodeterminewhetheragivendate,identifiedbyatickvalue,isstillpresentonamasterforreplication.BycallingtheseAPIs,clientscanmakeaninformeddecisionaboutwhetherthemastercanstillprovideallmissingdatastartingfromthepointuptowhichtheclienthadalreadysynchronized.Thiscanbehelpfulincaseareplicationclientisre-startedafterapause.

Masterserversnowalsotrackupthepointuptowhichtheyhavesentchangestoclientsforreplication.Thisinformationcanbeusedtodeterminethepointofdatathatreplicationclientshavereceivedfromthemaster,andifandhowfarapproximatelytheylagbehind.

Finally,restartingthereplicationapplieronaslaveserverhasbeenmademorerobustincasetheapplierwasstoppedwhiletherewerependingtransactionsonthemasterserver,andre-startingthereplicationapplierneedstorestorethestateofthesetransactions.

Clienttools

Thefilenamesindumpscreatedbyarangodumpnowcontainnotonlythenameofthedumpedcollection,butalsoanadditional32-digithashvalue.Thisisdonetopreventoverwritingdumpfilesincase-insensitivefilesystemswhenthereexistmultiplecollectionswiththesamename(butwithdifferentcases).

Forexample,ifadatabasehadtwocollectionstestandTest,previousversionsofarangodumpcreatedthefollowingfiles:

test.structure.jsonandtest.data.jsonforcollectiontestTest.structure.jsonandTest.data.jsonforcollectionTest

Thisdidnotworkincase-insensitivefilesystems,becausethefilesforthesecondcollectionwouldhaveoverwrittenthefilesofthefirst.arangodumpin2.7willcreatetheuniquefilesinthiscase,byappendingthe32-digithashvaluetothecollectionnameinallcase.Thesefilenameswillbeunambiguousevenincase-insensitivefilesystems.

WhatsNewin2.7

Bettercontrol-Csupportinarangosh

WhenCTRL-Cispressedinarangosh,itwillnowabortthelocallyrunningcommand(ifany).Ifnocommandwasrunning,pressingCTRL-Cwillprinta Cfirst.PressingCTRL-Cagainwillthenquitarangosh.

CTRL-Ccanalsobeusedtoresetthecurrentpromptwhileenteringcomplexnestedobjectswhichspanmultipleinputlines.

CTRL-CsupporthasbeenaddedtotheArangoShellversionsbuiltwithReadline-support(LinuxandMacOSonly).TheWindowsversionofArangoDBusesadifferentlibraryforhandlinginput,andsupportforCTRL-Chasnotbeenaddedthereyet.

Start/stop

LinuxstartupscriptsandsystemdconfigurationforarangodnowtrytoadjusttheNOFILE(numberofopenfiles)limitsfortheprocess.Thelimitvalueissetto131072(128k)whenArangoDBisstartedviastart/stopcommands.

ThiswillpreventarangodrunningoutofavailablefiledescriptorsincaseofmanyparallelHTTPconnectionsorlargecollectionswithmanydatafiles.

Additionally,whenArangoDBisstarted/stoppedmanuallyviathestart/stopcommands,themainprocesswillwaitforupto10secondsafteritforksthesupervisorandarangodchildprocesses.Ifthestartupfailswithinthatperiod,thestart/stopscriptwillfailwithanon-zeroexitcode,allowinganyinvokingscriptstohandlethiserror.Previousversionsalwaysreturnedanexitcodeof0,evenwhenarangodcouldn'tbestarted.

Ifthestartupofthesupervisororarangodisstillongoingafter10seconds,themainprogramwillstillreturnwithexitcode0inordertonotblockanyscripts.Thelimitof10secondsisarbitrarybecausethetimerequiredforanarangodstartupisnotknowninadvance.

Non-sparselogfiles

WALlogfilesanddatafilescreatedbyarangodarenownon-sparse.ThispreventsSIGBUSsignalsbeingraisedwhenamemory-mappedregionbackedbyasparsedatafilewasaccessedandthememoryregionwasnotactuallybackedbydisk,forexamplebecausethediskranoutofspace.

arangodnowalwaysfullyallocatesthediskspacerequiredforalogfileordatafilewhenitcreatesone,sothememoryregioncanalwaysbebackedbydisk,andmemorycanbeaccessedwithoutSIGBUSbeingraised.

WhatsNewin2.7

AQLchanges

DISTINCTisnowakeywordinAQL.

AQLqueriesthatuseDISTINCT(inlower,upperormixedcase)asanidentifier(i.e.asavariable,acollectionnameorafunctionname)willstopworking.Tomakesuchqueriesworkingagain,eachoccurrenceofDISTINCTinanAQLqueryshouldbeenclosedinbackticks.ThiswillturnDISTINCTfromakeywordintoanidentifieragain.

TheAQLfunctionSKIPLIST()hasbeenremovedinArangoDB2.7.ThisfunctionwasdeprecatedinArangoDB2.6.Itwasaleft-overfromtimeswhenthequeryoptimizerwasn'tabletouseskiplistindexestogetherwithfilters,skipandlimitvalues.Sincethisissuebeenfixedsinceversion2.3,thereisnoAQLreplacementfunctionforSKIPLIST.QueriesthatusetheSKIPLISTfunctioncanbefixedbyusingtheusualcombinationofFOR,FILTERandLIMIT,e.g.

FILTERdoc.value>=@value

SORTdoc.valueDESC

LIMIT1

RETURNdoc

Foxxchanges

Bundlingandcompilation

TheassetspropertyisnolongersupportedinFoxxmanifestsandisscheduledtoberemovedinafutureversionofArangoDB.Thefilespropertycanstillbeusedtoservestaticassetsbutitisrecommendedtouseseparatetoolingtocompileandbundleyourassets.

Manifestscripts

Thepropertiessetupandteardownhavebeenmovedintothescriptspropertymap:

Before:

"teardown":"scripts/teardown.js"

After:

"scripts":{

"teardown":"scripts/teardown.js"

FoxxQueues

Function-basedFoxxQueuejobtypesarenolongersupported.Tolearnabouthowyoucanusethenewscript-basedjobtypesfollowtheupdatedrecipeinthecookbook.

FoxxSessions

ThejwtandtypeoptionshavebeenremovedfromtheactivateSessionsAPI.

IfyouwanttoreplicatethebehaviorofthejwtoptionyoucanusetheJWTfunctionsinthecryptomodule.AJWT-basedsessionstoragethatdoesn'twritesessionstothedatabaseisavailableasthesessions-jwtappintheFoxxappstore.

Thesessiontypeisnowinferredfromthepresenceofthecookieorheaderoptions(allowingyoutoenablesupportforboth).Ifyouwanttousethedefaultsettingsforcookieorheaderyoucanpassthevaluetrueinstead.

ThesessionStorageAppoptionhasbeenremovedinfavourofthesessionStorageoption.

Before:

varctrl=newFoxx.Controller(applicationContext);

sessionStorageApp:'some-sessions-app',

type:'cookie'

After:

sessionStorage:applicationContext.dependencies.sessions.sessionStorage,

cookie:true

Requestmodule

Themoduleorg/arangodb/requestusesaninternallibraryfunctionforsendingHTTPrequests.ThislibraryfunctionallyunconditionallysetanHTTPheaderAccept-Encoding:gzipinalloutgoingHTTPrequests,withoutclientcodehavingtosetthisheaderexplicitly.

Thishasbeenfixedin2.7,soAccept-Encoding:gzipisnotsetautomaticallyanymore.AdditionallytheheaderUser-Agent:ArangoDBisnotsetautomaticallyeither.Ifclientapplicationsrelyontheseheadersbeingsent,theyarefreetoadditwhenconstructingrequestsusingtherequestmodule.

Theinternal.download()functionisalsoaffectedbythischange.Again,theheadercanbeaddedhereifrequiredbypassingitviaaheaderssub-attributeinthethirdparameter(options)tothisfunction.

arangodump/backupsThefilenamesindumpscreatedbyarangodumpnowcontainnotonlythenameofthedumpedcollection,butalsoanadditional32-digithashvalue.Thisisdonetopreventoverwritingdumpfilesincase-insensitivefilesystemswhenthereexistmultiplecollectionswiththesamename(butwithdifferentcases).

Thischangeleadstochangedfilenamesindumpscreatedbyarangodump.Ifanyclientscriptsdependonthefilenamesinthedumpoutputdirectorybeingequaltothecollectionnameplusoneofthesuffixes.structure.jsonand.data.json,theyneedtobeadjusted.

StartingwithArangoDB2.7,thefilenameswillcontainanunderscoreplusthe32-digitMD5value(representedinhexadecimalnotation)ofthecollectionname.

Forexample,whenarangodumpdumpsdataoftwocollectionstestandTest,thefilenamesinpreviousversionsofArangoDBwere:

test.structure.json(definitionsforcollectiontest)test.data.json(dataforcollectiontest)Test.structure.json(definitionsforcollectionTest)Test.data.json(dataforcollectionTest)

In2.7,thefilenameswillbe:

test_098f6bcd4621d373cade4e832627b4f6.structure.json(definitionsforcollectiontest)

test_098f6bcd4621d373cade4e832627b4f6.data.json(dataforcollectiontest)Test_0cbc6611f5540bd0809a388dc95a615b.structure.json(definitionsforcollectionTest)Test_0cbc6611f5540bd0809a388dc95a615b.data.json(dataforcollectionTest)

Starting/stopping

Whenstartingarangod,theserverwillnowdroptheprocessprivilegestothespecifiedvaluesinoptions--server.uidand--server.gidinstantlyafterparsingthestartupoptions.

Thatmeanswheneither--server.uidor--server.gidareset,theprivilegechangewillhappenearlier.Thismaypreventbindingtheservertoanendpointwithaportnumberlowerthan1024ifthearangodbuserhasnoprivilegesforthat.PreviousversionsofArangoDBchangedtheprivilegeslater,sosomestartupactionswerestillcarriedoutundertheinvokinguser(i.e.likelyrootwhenstartedviainit.dorsystemscripts)andespeciallybindingtolowportnumberswasstillpossiblethere.

Thedefaultprivilegesforuserarangodbwillnotbesufficientforbindingtoportnumberslowerthan1024.TohaveanArangoDB2.7bindtoaportnumberlowerthan1024,itneedstobestartedwitheitheradifferentprivilegeduser,ortheprivilegesofthearangodbuserhavetoraisedmanuallybeforehand.

Additionally,LinuxstartupscriptsandsystemdconfigurationforarangodnowwilladjusttheNOFILE(numberofopenfiles)limitsfortheprocess.Thelimitvalueissetto131072(128k)whenArangoDBisstartedviastart/stopcommands.Thegoalofthischangeistopreventarangodfromrunningoutofavailablefiledescriptorsforsocketconnectionsanddatafiles.

Connectionhandling

arangodwillnowactuallycloselingeringclientconnectionswhenidleforatleastthedurationspecifiedinthe--server.keep-alive-timeoutstartupoption.

InpreviousversionsofArangoDB,idleconnectionswerenotclosedbytheserverwhenthetimeoutwasreachedandtheclientwasstillconnected.Nowtheconnectionisproperlyclosedbytheserverincaseoftimeout.Clientapplicationsrelyingontheoldbehaviormaynowneedtoreconnecttotheserverwhentheiridleconnectionstimeoutandgetclosed(note:connectionsbeingidleforalongtimemaybeclosedbytheOSorfirewallsanyway-clientapplicationsshouldbeawareofthatandtrytoreconnect).

Optionchanges

Configureoptionsremoved

Thefollowingoptionsforconfigurehavebeenremovedbecausetheywereunusedorexotic:

--enable-timings

--enable-figures

Startupoptionsadded

Thefollowingconfigurationoptionshavebeenaddedin2.7:

--database.query-cache-max-results:setsthemaximumnumberofresultsinAQLqueryresultcacheperdatabase--database.query-cache-mode:setsthemodefortheAQLqueryresultscache.Possiblevaluesareon,offanddemand.Thedefaultvalueisoff

Simplequeries

Manysimplequeriesprovideaskip()functionthatcanbeusedtoskipoveracertainnumberofdocumentsintheresult.ThisfunctionallowedspecifyingnegativeoffsetsinpreviousversionsofArangoDB.Specifyinganegativeoffsetledtothequeryresultbeingiteratedinreverseorder,soskippingwasperformedfromthebackoftheresult.Asmostsimplequeriesdonotprovideaguaranteedresultorder,

skippingfromthebackofaresultwithunspecificorderseemsaratherexoticusecaseandwasremovedtoincreaseconsistencywith

AQL,whichalsodoesnotprovidenegativeskipvalues.

NegativeskipvaluesweredeprecatedinArangoDB2.6.

TasksAPI

TheundocumentedfunctionaddJob()hasbeenremovedfromtheorg/arangodb/tasksmoduleinArangoDB2.7.

RuntimeendpointsmanipulationAPI

ThefollowingHTTPRESTAPImethodsforruntimemanipulationofserverendpointshavebeenremovedinArangoDB2.7:

POST/_api/endpoint:todynamicallyaddanendpointwhiletheserverwasrunningDELETE/_api/endpoint:todynamicallyremoveanendpointwhiletheserverwasrunning

ThischangealsoaffectstheequivalentJavaScriptendpointmanipulationmethodsavailableinFoxx.ThefollowingfunctionshavebeenremovedinArangoDB2.7:

db._configureEndpoint()

db._removeEndpoint()

APIsadded

Batchdocumentremovalandlookupcommands

Thefollowingcommandshavebeenaddedforcollectionobjects:

collection.lookupByKeys(keys)collection.removeByKeys(keys)

Thesecommandscanbeusedtoperformmulti-documentlookupandremovaloperationsefficientlyfromtheArangoShell.Theargumenttotheseoperationsisanarrayofdocumentkeys.

ThesecommandscanalsobeusedviatheHTTPRESTAPI.Theirendpointsare:

PUT/_api/simple/lookup-by-keysPUT/_api/simple/remove-by-keys

CollectionexportHTTPRESTAPI

ArangoDBnowprovidesadedicatedcollectionexportAPI,whichcantakesnapshotsofentirecollectionsmoreefficientlythanthegeneral-purposecursorAPI.TheexportAPIisusefultotransferthecontentsofanentirecollectiontoaclientapplication.Itprovidesoptionalfilteringonspecificattributes.

TheexportAPIisavailableatendpointPOST/_api/export?collection=....TheAPIhasthesamereturnvaluestructureasthealreadyestablishedcursorAPI(POST/_api/cursor).

AnintroductiontotheexportAPIisgiveninthisblogpost:http://jsteemann.github.io/blog/2015/04/04/more-efficient-data-exports/

AQLimprovements

EDGESAQLFunction

TheAQLfunctionEDGESgotanewfifthoptionalparameter,whichmustbeanobjectifspecified.Rightnowonlyoneoptionisavailableforit:

includeVerticesthisisabooleanparameterthatallowstomodifytheresultofEDGES().ThedefaultvalueforincludeVerticesisfalse,whichdoesnothaveanyeffect.Settingittotruewillmodifytheresult,suchthatalsotheconnectedverticesarereturnedalongwiththeedges:

{vertex:<vertexDocument>,edge:<edgeDocument>}

SubqueryoptimizationsforAQLqueries

Thisoptimizationavoidscopyingintermediateresultsintosubqueriesthatarenotrequiredbythesubquery.

Abriefdescriptioncanbefoundhere:http://jsteemann.github.io/blog/2015/05/04/subquery-optimizations/

ReturnvalueoptimizationforAQLqueries

Thisoptimizationavoidscopyingthefinalqueryresultinsidethequery'smainReturnNode.

Abriefdescriptioncanbefoundhere:http://jsteemann.github.io/blog/2015/05/04/return-value-optimization-for-aql/

WhatsNewin2.6

SpeedupAQLqueriescontainingbigINlistsforindexlookups

INlistsusedforindexlookupshadperformanceissuesinpreviousversionsofArangoDB.Theseissueshavebeenaddressedin2.6sousingbiggerINlistsforfilteringismuchfaster.

Abriefdescriptioncanbefoundhere:http://jsteemann.github.io/blog/2015/05/07/in-list-improvements/

AddedalternativeimplementationforAQLCOLLECT

Thealternativemethodusesahashtableforgroupinganddoesnotrequireitsinputelementstobesorted.ItwillbetakenintoaccountbytheoptimizerforCOLLECTstatementsthatdonotuseanINTOclause.

IncaseaCOLLECTstatementcanusethehashtablevariant,theoptimizerwillcreateanextraplanforitatthebeginningoftheplanningphase.Inthisplan,noextraSORTnodewillbeaddedinfrontoftheCOLLECTbecausethehashtablevariantofCOLLECTdoesnotrequiresortedinput.Instead,aSORTnodewillbeaddedafterittosortitsoutput.ThisSORTnodemaybeoptimizedawayagaininlaterstages.Ifthesortorderoftheresultisirrelevanttotheuser,addinganextraSORTnullafterahashCOLLECToperationwillallowtheoptimizertoremovethesortsaltogether.

InadditiontothehashtablevariantofCOLLECT,theoptimizerwillmodifytheoriginalplantousetheregularCOLLECTimplementation.Asthisimplementationrequiressortedinput,theoptimizerwillinsertaSORTnodeinfrontoftheCOLLECT.ThisSORTnodemaybeoptimizedawayinlaterstages.

Thecreatedplanswillthenbeshippedthroughtheregularoptimizationpipeline.Intheend,theoptimizerwillpicktheplanwiththelowestestimatedtotalcostasusual.Thehashtablevariantdoesnotrequireanup-frontsortoftheinput,andwillthusbepreferredovertheregularCOLLECTiftheoptimizerestimatesmanyinputelementsfortheCOLLECTnodeandcannotuseanindextosortthem.

TheoptimizercanbeexplicitlytoldtousetheregularsortedvariantofCOLLECTbysuffixingaCOLLECTstatementwithOPTIONS{"method":"sorted"}.ThiswilloverridetheoptimizerguessworkandonlyproducethesortedvariantofCOLLECT.

AblogpostonthenewCOLLECTimplementationcanbefoundhere:http://jsteemann.github.io/blog/2015/04/22/collecting-with-a-hash-table/

Simplifiedreturnvaluesyntaxfordata-modificationAQLqueries

ArangoDB2.4sinceversionallowstoreturnresultsfromdata-modificationAQLqueries.Thesyntaxforthiswasquitelimitedandverbose:

FORiIN1..10

INSERT{value:i}INtest

LETinserted=NEW

RETURNinserted

TheLETinserted=NEWRETURNinsertedwasrequiredliterallytoreturntheinserteddocuments.Nocalculationscouldbemadeusingtheinserteddocuments.

Thisisnowmoreflexible.Afteradata-modificationclause(e.g.INSERT,UPDATE,REPLACE,REMOVE,UPSERT)therecanfollowanynumberofLETcalculations.Thesecalculationscanrefertothepseudo-valuesOLDandNEWthatarecreatedbythedata-modificationstatements.

Thisallowsreturningprojectionsofinsertedorupdateddocuments,e.g.:

FORiIN1..10

INSERT{value:i}INtest

RETURN{_key:NEW._key,value:i}

Stillnoteveryconstructisallowedafteradata-modificationclause.Forexample,nofunctionscanbecalledthatmayaccessdocuments.

Moreinformationcanbefoundhere:http://jsteemann.github.io/blog/2015/03/27/improvements-for-data-modification-queries/

AddedAQLUPSERTstatement

WhatsNewin2.6

ThisaddsanUPSERTstatementtoAQLthatisacombinationofbothINSERTandUPDATE/REPLACE.TheUPSERTwillsearchforamatchingdocumentusingauser-providedexample.Ifnodocumentmatchestheexample,theinsertpartoftheUPSERTstatementwillbeexecuted.Ifthereisamatch,theupdate/replacepartwillbecarriedout:

UPSERT{page:'index.html'}/*searchexample*/

INSERT{page:'index.html',pageViews:1}/*insertpart*/

UPDATE{pageViews:OLD.pageViews+1}/*updatepart*/

INpageViews

UPSERTcanbeusedwithanUPDATEorREPLACEclause.TheUPDATEclausewillperformapartialupdateofthefounddocument,whereastheREPLACEclausewillreplacethefounddocumententirely.TheUPDATEorREPLACEpartscanrefertothepseudo-valueOLD,whichcontainsallattributesofthefounddocument.

UPSERTstatementscanoptionallyreturnvalues.Inthefollowingquery,thereturnattributefoundwillreturnthefounddocumentbeforetheUPDATEwasapplied.Ifnodocumentwasfound,foundwillcontainavalueofnull.Theupdatedresultattributewillcontaintheinserted/updateddocument:

UPSERT{page:'index.html'}/*searchexample*/

INSERT{page:'index.html',pageViews:1}/*insertpart*/

UPDATE{pageViews:OLD.pageViews+1}/*updatepart*/

INpageViews

RETURN{found:OLD,updated:NEW}

AmoredetaileddescriptionofUPSERTcanbefoundhere:http://jsteemann.github.io/blog/2015/03/27/preview-of-the-upsert-command/

WhenerrorsoccurinsideAQLuserfunctions,theerrormessagewillnowcontainastacktrace,indicatingthelineofcodeinwhichtheerroroccurred.ThisshouldmakedebuggingAQLuserfunctionseasier.

WebAdminInterfaceArangoDB'sbuilt-inwebinterfacenowusessessions.Sessioninformationisstoredincookies,soclientsusingthewebinterfacemustacceptcookiesinordertouseit.

Thenewstartupoption--server.session-timeoutcanbeusedforadjustingthesessionlifetime.

TheAQLeditorinthewebinterfacenowprovidesanexplainfunctionality,whichcanbeusedforinspectingandperformance-tuningAQLqueries.ThequeryexecutiontimeisnowalsodisplayedintheAQLeditor.

Foxxappsthatrequireconfigurationoraremissingdependenciesarenowindicatedintheappoverviewanddetails.

Foxximprovements

ConfigurationandDependencies

Foxxappmanifestscannowdefineconfigurationoptions,aswellasdependenciesonotherFoxxapps.

AnintroductiontoFoxxconfigurationscanbefoundintheblog:https://www.arangodb.com/2015/05/reusable-foxx-apps-with-configurations/

AndtheblogpostonFoxxdependenciescanbefoundhere:https://www.arangodb.com/2015/05/foxx-dependencies-for-more-composable-foxx-apps/

MochaTests

YoucannowwritetestsforyourFoxxappsusingtheMochatestingframework:https://www.arangodb.com/2015/04/testing-foxx-mocha/

ArecipeforwritingtestsforyourFoxxappscanbefoundinthecookbook:https://docs.arangodb.com/2.8/cookbook/FoxxTesting.html

WhatsNewin2.6

APIDocumentation

TheAPIdocumentationhasbeenupdatedtoSwagger2.YoucannowalsomountAPIdocumentationinyourownFoxxapps.

Alsoseetheblogpostintroducingthisfeature:https://www.arangodb.com/2015/05/document-your-foxx-apps-with-swagger-2/

CustomScriptsandFoxxQueue

InadditiontotheexistingsetupandteardownscriptsyoucannowdefinecustomscriptsinyourFoxxmanifestandinvoketheseusingthewebadmininterfaceortheFoxxmanagerCLI.Thesescriptscannowalsotakepositionalargumentsandexportreturnvalues.

JobtypesfortheFoxxQueuecannowbedefinedasascriptnameandappmountpathallowingtheuseofFoxxscriptsasjobtypes.Thepre-2.6jobtypesareknowntocauseissueswhenrestartingtheserverandareerror-prone;westronglyrecommendedconvertinganyexistingjobtypestothenewformat.

Clienttools

Thedefaultconfigurationvaluefortheoption--server.request-timeoutwasincreasedfrom300to1200secondsforallclienttools(arangosh,arangoimp,arangodump,arangorestore).

Thedefaultconfigurationvaluefortheoption--server.connect-timeoutwasincreasedfrom3to5secondsforclienttools(arangosh,arangoimp,arangodump,arangorestore).

Arangorestore

Theoption--create-databasewasaddedforarangorestore.

Settingthisoptiontotruewillnowcreatethetargetdatabaseifitdoesnotexist.Whencreatingthetargetdatabase,theusernameandpasswordspassedtoarangorestorewillbeusedtocreateaninitialuserforthenewdatabase.

Thedefaultvalueforthisoptionisfalse.

Arangoimp

Arangoimpcannowoptionallyupdateorreplaceexistingdocuments,providedtheimportdatacontainsdocumentswith_keyattributes.

Previously,theimportcouldbeusedforinsertingnewdocumentsonly,andre-insertingadocumentwithanexistingkeywouldhavefailedwithauniquekeyconstraintviolatederror.

Thebehaviorofarangoimp(insert,update,replaceonduplicatekey)cannowbecontrolledwiththeoption--on-duplicate.Theoptioncanhaveoneofthefollowingvalues:

error:whenauniquekeyconstrainterroroccurs,donotimportorupdatethedocumentbutreportanerror.Thisisthedefault.

update:whenauniquekeyconstrainterroroccurs,tryto(partially)updatetheexistingdocumentwiththedataspecifiedintheimport.Thismaystillfailifthedocumentwouldviolatesecondaryuniqueindexes.Onlytheattributespresentintheimportdatawillbeupdatedandotherattributesalreadypresentwillbepreserved.ThenumberofupdateddocumentswillbereportedintheupdatedattributeoftheHTTPAPIresult.

replace:whenauniquekeyconstrainterroroccurs,trytofullyreplacetheexistingdocumentwiththedataspecifiedintheimport.Thismaystillfailifthedocumentwouldviolatesecondaryuniqueindexes.ThenumberofreplaceddocumentswillbereportedintheupdatedattributeoftheHTTPAPIresult.

ignore:whenauniquekeyconstrainterroroccurs,ignorethiserror.Therewillbenoinsert,updateorreplacefortheparticulardocument.IgnoreddocumentswillbereportedseparatelyintheignoredattributeoftheHTTPAPIresult.

Thedefaultvalueiserror.

Afewexamplesforusingarangoimpwiththe--on-duplicateoptioncanbefoundhere:http://jsteemann.github.io/blog/2015/04/14/updating-documents-with-arangoimp/

WhatsNewin2.6

SomeLinux-basedArangoDBpackagesarenowusingtcmallocformemoryallocator.

UpgradedICUlibrarytoversion54.Thisincreasesperformanceinmanyplaces.

Allowtosplitanedgeindexintobucketswhichareresizedindividually.Thedefaultvalueis1,resemblingthepre-2.6behavior.Usingmultiplebucketswillleadtotheindexentriesbeingdistributedtotheindividualbuckets,witheachbucketbeingresponsibleonlyforafractionofthetotalindexentries.Usingmultiplebucketsmayleadtomorefrequentbutmuchfasterindexbucketresizes,andisrecommendedforbiggeredgecollections.

Defaultconfigurationvalueforoption--server.backlog-sizewaschangedfrom10to64.

Defaultconfigurationvalueforoption--database.ignore-datafile-errorswaschangedfromtruetofalse

Documentkeyscannowcontain@and.characters

Fulltextindexcannowindextextvaluescontainedindirectsub-objectsoftheindexedattribute.

PreviousversionsofArangoDBonlyindexedtheattributevalueifitwasastring.Sub-attributesoftheindexattributewereignoredwhenfulltextindexing.

Now,iftheindexattributevalueisanobject,theobject'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.Iftheindexattributevalueisanarray,thearray'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.

Forexample,withafulltextindexpresentonthetranslationsattribute,thefollowingtextvalueswillnowbeindexed:

varc=db._create("example");

c.ensureFulltextIndex("translations");

c.insert({translations:{en:"fox",de:"Fuchs",fr:"renard",ru:"лиса"}});

c.insert({translations:"FoxistheEnglishtranslationoftheGermanwordFuchs"});

c.insert({translations:["ArangoDB","document","database","Foxx"]});

c.fulltext("translations","лиса").toArray();//returnsonlyfirstdocument

c.fulltext("translations","Fox").toArray();//returnsfirstandseconddocuments

c.fulltext("translations","prefix:Fox").toArray();//returnsallthreedocuments

Addedconfigurationoption--server.foxx-queues-poll-interval

ThisstartupoptioncontrolsthefrequencywithwhichtheFoxxqueuesmanagerischeckingthequeue(orqueues)forjobstobeexecuted.

Thedefaultvalueis1second.Loweringthisvaluewillresultinthequeuemanagerwakingupandcheckingthequeuesmorefrequently,whichmayincreaseCPUusageoftheserver.WhennotusingFoxxqueues,thisvaluecanberaisedtosavesomeCPUtime.

Addedconfigurationoption--server.foxx-queues

ThisstartupoptioncontrolswhethertheFoxxqueuemanagerwillcheckqueueandjobentriesinthe_systemdatabaseonly.RestrictingtheFoxxqueuemanagertothe_systemdatabasewillleadtothequeuemanagerhavingtocheckonlythequeuescollectionofasingledatabase,whereasmakingitcheckthequeuesofalldatabasesmightresultinmoreworktobedoneandmoreCPUtimetobeusedbythequeuemanager.

WhatsNewin2.6

Requirements

ArangoDB'sbuilt-inwebinterfacenowusescookiesforsessionmanagement.Sessioninformationidsarestoredincookies,soclientsusingthewebinterfacemustacceptcookiesinordertologinanduseit.

Foxxchanges

FoxxQueues

FoxxQueuejobtypedefinitionswerepreviouslybasedonfunctionsandhadtoberegisteredbeforeuse.Duetochangesin2.5thisresultedinproblemswhenrestartingtheserverordefiningjobtypesincorrectly.

Function-basedjobtypeshavebeendeprecatedin2.6andwillberemovedentirelyin2.7.

Inordertoconvertexistingfunction-basedjobtypestothenewscript-basedjobtypes,createcustomscriptsinyourFoxxappandreferencethembytheirnameandthemountpointoftheapptheyaredefinedin.OfficialjobtypesfromtheFoxxappstorecanbeupgradedbyupgradingfromthe1.xversiontothe2.xversionofthesameapp.

Inordertoupgradequeuedjobstothenewjobtypes,youneedtoupdatethetypepropertyoftheaffectedjobsinthedatabase's_jobssystemcollection.Inordertoseethecollectioninthewebinterfaceyouneedtoenablethecollectiontype"System"inthecollectionlistoptions.

Example:

Before:"type":"mailer.postmark"

After:"type":{"name":"mailer","mount":"/my-postmark-mailer"}

FoxxSessions

Theoptionsjwtandtypeofthecontrollermethodcontroller.activateSessionshavebeendeprecatedin2.6andwillberemovedentirelyin2.7.

IfyouwanttousepureJWTsessions,youcanusethesessions-jwtFoxxappfromtheFoxxappstore.

IfyouwanttouseyourownJWT-basedsessions,youcanusetheJWTfunctionsinthecryptomoduledirectly.

Insteadofusingthetypeoptionyoucanjustusethecookieandheaderoptionsontheirown,whichbothnowacceptthevaluetruetoenablethemwiththeirdefaultconfigurations.

TheoptionsessionStorageApphasbeenrenamedtosessionStorageandnowalsoacceptssessionstoragesdirectly.TheoldoptionsessionStorageAppwillberemovedentirelyin2.7.

Libraries

ThebundledversionofthejoilibraryusedinFoxxwasupgradedtoversion6.0.8.ThismayaffectFoxxapplicationsthatdependonthelibrary.

AQLchanges

AQLLENGTHfunction

ThereturnvalueoftheAQLLENGTHfunctionwaschangedifLENGTHisappliedonnullorabooleanvalue:

LENGTH(null)nowreturns0.InpreviousversionsofArangoDB,thisreturned4.

LENGTH(false)nowreturns0.InpreviousversionsofArangoDB,thereturnvaluewas5.

LENGTH(true)nowreturns1.InpreviousversionsofArangoDB,thereturnvaluewas4.

AQLgraphfunctions

In2.6thegraphfunctionsdidundergoaperformancelifting.Duringthisprocesswehadtoadopttheresultformatandtheoptionsforsomeofthem.ManygraphfunctionsnowhaveanoptionincludeDatawhichallowstotriggeriftheresultofthisfunctionshouldcontainfullyextracteddocumentsincludeData:trueoronlythe_idvaluesincludeData:false.Inmostusecasesthe_idissufficienttocontinueandtheextractionofdataisanunnecessaryoperation.TheAQLfunctionssupportingthisadditionaloptionare:

SHORTEST_PATHNEIGHBORSGRAPH_SHORTEST_PATHGRAPH_NEIGHBORSGRAPH_EDGES

FurthermoretheresultSHORTEST_PATHhaschanged.Theoldformatreturnedalistofallverticesonthepath.Optionallyitcouldincludeeachsub-pathforthesevertices.Allofthedocumentswerefullyextracted.Example:

vertex:{

_id:"vertex/1",

_key:"1",

_rev:"1234"

name:"Alice"

path:{

vertices:[

_id:"vertex/1",

_key:"1",

_rev:"1234"

name:"Alice"

edges:[]

vertex:{

_id:"vertex/2",

_key:"2",

_rev:"5678"

name:"Bob"

path:{

vertices:[

_id:"vertex/1",

_key:"1",

_rev:"1234"

name:"Alice"

_id:"vertex/2",

_key:"2",

_rev:"5678"

name:"Bob"

edges:[

_id:"edge/1",

_key:"1",

_rev:"9876",

type:"loves"

Thenewversionismorecompact.EachSHORTEST_PATHwillonlyreturnonedocumenthavingtheattributesvertices,edges,distance.Thedistanceiscomputedtakingintoaccountthegivenweight.OptionallythedocumentscanbeextractedwithincludeData:trueExample:

vertices:[

"vertex/1",

"vertex/2"

edges:[

"edge/1"

distance:1

ThenextfunctionthatreturnsadifferentformatisNEIGHBORS.Since2.5itreturnedanobjectwithedgeandvertexforeachconnectededge.Example:

vertex:{

_id:"vertex/2",

_key:"2",

_rev:"5678"

name:"Bob"

edge:{

_id:"edge/1",

_key:"1",

_rev:"9876",

type:"loves"

With2.6itwillonlyreturnthevertexdirectly,againusingincludeData:true.Bydefaultitwillreturnadistinctsetofneighbors,usingtheoptiondistinct:falsewillincludethesamevertexforeachedgepointingtoit.

Example:

"vertex/2"

FunctionandAPIchanges

Graphmeasurementsfunctions

AllgraphmeasurementsfunctionsinJavaScriptmodulegeneral-graphthatcalculatedasinglefigurepreviouslyreturnedanarraycontainingjustthefigure.Nowthesefunctionswillreturnthefiguredirectlyandnotputitinsideanarray.

Theaffectedfunctionsare:

graph._absoluteEccentricity

graph._eccentricity

graph._absoluteCloseness

graph._closeness

graph._absoluteBetweenness

graph._betweenness

graph._radius

graph._diameter

Clientprogramscallingthesefunctionsshouldbeadjustedsotheyprocessthescalarvaluereturnedbythefunctioninsteadofthepreviousarrayvalue.

CursorAPI

AbatchSizevalue0isnowdisallowedwhencallingthecursorAPIviaHTTPPOST/_api/cursor.

TheHTTPRESTAPIPOST/_api/cursordoesnotacceptabatchSizeparametervalueof0anylonger.Abatchsizeof0nevermademuchsense,butpreviousversionsofArangoDBdidnotcheckforthisvalue.NowcreatingacursorusingabatchSizevalue0willresultinanHTTP400errorresponse.

DocumentURLsreturned

TheRESTAPImethodGET/_api/document?collection=...(thatmethodwillreturnpartialURLstoalldocumentsinthecollection)willnowproperlyprefixdocumentaddressURLswiththecurrentdatabasename.

PreviousversionsofArangoDBreturnedtheURLsstartingwith/_api/butwithoutthecurrentdatabasename,e.g./_api/document/mycollection/mykey.Startingwith2.6,theresponseURLswillincludethedatabasenameaswell,e.g./_db/_system/_api/document/mycollection/mykey.

Fulltextindexing

Fulltextindexeswillnowalsoindextextvaluescontainedindirectsub-objectsoftheindexedattribute.

PreviousversionsofArangoDBonlyindexedtheattributevalueifitwasastring.Sub-attributesoftheindexattributewereignoredwhenfulltextindexing.

Now,iftheindexattributevalueisanobject,theobject'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.Iftheindexattributevalueisanarray,thearray'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.

Deprecatedserverfunctionality

Simplequeries

Thefollowingsimplequeryfunctionsarenowdeprecated:

collection.nearcollection.withincollection.geocollection.fulltextcollection.rangecollection.closedRange

ThisalsoleadtothefollowingRESTAPImethodsbeingdeprecatedfromnowon:

PUT/_api/simple/nearPUT/_api/simple/withinPUT/_api/simple/fulltextPUT/_api/simple/range

ItisrecommendedtoreplacecallstothesefunctionsorAPIswithequivalentAQLqueries,whicharemoreflexiblebecausetheycanbecombinedwithotheroperations:

RETURNdoc

FORdocINFULLTEXT(@@collection,@attributeName,@queryString,@limit)

RETURNdoc

FILTERdoc.value>=@left&&doc.value<@right

LIMIT@skip,@limit

RETURNdoc`

TheabovesimplequeryfunctionsandRESTAPImethodsmayberemovedinfutureversionsofArangoDB.

UsingnegativevaluesforSimpleQuery.skip()isalsodeprecated.ThisfunctionalitywillberemovedinfutureversionsofArangoDB.

AQLfunctions

TheAQLSKIPLISTfunctionhasbeendeprecatedbecauseitisobsolete.

ThefunctionwasintroducedinolderversionsofArangoDBwithalesspowerfulqueryoptimizertoretrievedatafromaskiplistindexusingaLIMITclause.

Since2.3thesamegoalcanbeachievedbyusingregularAQLconstructs,e.g.

FILTERdoc.value>=@value

SORTdoc.value

LIMIT1

RETURNdoc

Startupoptionchanges

Optionsadded

Thefollowingconfigurationoptionshavebeenaddedin2.6:

--server.session-timeout:allowscontrollingthetimeoutofusersessionsinthewebinterface.Thevalueisspecifiedinseconds.

--server.foxx-queues:controlswhethertheFoxxqueuemanagerwillcheckqueueandjobentries.DisablingthisoptioncanreduceserverloadbutwillpreventjobsaddedtoFoxxqueuesfrombeingprocessedatall.

Thedefaultvalueistrue,enablingtheFoxxqueuesfeature.

--server.foxx-queues-poll-interval:allowsadjustingthefrequencywithwhichtheFoxxqueuesmanagerischeckingthequeue(orqueues)forjobstobeexecuted.

Thedefaultvalueis1second.Loweringthisvaluewillresultinthequeuemanagerwakingupandcheckingthequeuesmorefrequently,whichmayincreaseCPUusageoftheserver.

Note:thisoptiononlyhasaneffectwhen--server.foxx-queuesisnotsettofalse.

Optionsremoved

Thefollowingconfigurationoptionshavebeenremovedin2.6.:

--log.severity:thedocsfor--log.severitymentionedlotsofseverities(e.g.exception,technical,functional,development)butonlyafewseverities(e.g.all,human)wereactuallyused,withhumanbeingthedefaultandallenablingtheadditionalloggingofincomingrequests.

Theoptionpretendedtocontrolalotofthingswhichitactuallydidn't.Additionally,theoption--log.requests-filewasaroundforalongtimealready,alsocontrollingrequestlogging.

Becausethe--log.severityoptioneffectivelydidnotcontrolthatmuch,itwasremoved.Asideeffectofremovingtheoptionisthat2.5installationsstartedwithoption--log.severityallwillnotlogrequestsaftertheupgradeto2.6.Thiscanbeadjustedbysettingthe--log.requests-fileoptioninstead.

Defaultvalueschanged

Thedefaultvaluesforthefollowingoptionshavechangedin2.6:

--database.ignore-datafile-errors:thedefaultvalueforthisoptionwaschangedfromtruetofalse.

Ifthenewdefaultvalueoffalseisused,thenarangodwillrefuseloadingcollectionsthatcontaindatafileswithCRCmismatchesorothererrors.Acollectionwithdatafileerrorswillthenbecomeunavailable.Thispreventsfollowuperrorsfromhappening.

Theonlywaytoaccesssuchcollectionistousethedatafiledebugger(arango-dfdb)andtrytorepairortruncatethedatafilewithit.

--server.request-timeout:thedefaultvaluewasincreasedfrom300to1200secondsforallclienttools(arangosh,arangoimp,arangodump,arangorestore).

--server.connect-timeout:thedefaultvaluewasincreasedfrom3to5secondsforallclienttools(arangosh,arangoimp,arangodump,arangorestore).

V8versionupgrade

Thebuilt-inversionofV8hasbeenupgradedfrom3.29.54to3.31.74.1.ThisallowsactivatingadditionalES6(alsodubbedHarmonyorES.next)featuresinArangoDB,bothintheArangoShellandtheArangoDBserver.Theycanbeusedforscriptingandinserver-sideactionssuchasFoxxroutes,traversalsetc.

ThefollowingadditionalES6featuresbecomeavailableinArangoDB2.5bydefault:

iteratorsandgeneratorstemplatestringsenhancedobjectliteralsenhancednumericliteralsblockscopingwithletandconstantvariablesusingconst(note:constantvariablesrequireusingstrictmode,too)additionalstringmethods(suchasstartsWith,repeatetc.)

Indeximprovements

Sparsehashandskiplistindexes

Hashandskiplistindexescanoptionallybemadesparse.Sparseindexesexcludedocumentsinwhichatleastoneoftheindexattributesiseithernotsetorhasavalueofnull.

Assuchdocumentsareexcludedfromsparseindexes,theymaycontainfewerdocumentsthantheirnon-sparsecounterparts.Thisenablesfasterindexingandcanleadtoreducedmemoryusageincasetheindexedattributedoesoccuronlyinsome,butnotalldocumentsofthecollection.Sparseindexeswillalsoreducethenumberofcollisionsinnon-uniquehashindexesincasenon-existingoroptionalattributesareindexed.

Inordertocreateasparseindex,anobjectwiththeattributesparsecanbeaddedtotheindexcreationcommands:

db.collection.ensureHashIndex(attributeName,{sparse:true});

db.collection.ensureHashIndex(attributeName1,attributeName2,{sparse:true});

db.collection.ensureUniqueConstraint(attributeName,{sparse:true});

db.collection.ensureUniqueConstraint(attributeName1,attributeName2,{sparse:true});

db.collection.ensureSkiplist(attributeName,{sparse:true});

db.collection.ensureSkiplist(attributeName1,attributeName2,{sparse:true});

db.collection.ensureUniqueSkiplist(attributeName,{sparse:true});

db.collection.ensureUniqueSkiplist(attributeName1,attributeName2,{sparse:true});

Notethatinplaceoftheabovespecializedindexcreationcommands,itisrecommendedtousethemoregeneralindexcreationcommandensureIndex:

db.collection.ensureIndex({type:"hash",sparse:true,unique:true,fields:[attributeName]});

db.collection.ensureIndex({type:"skiplist",sparse:false,unique:false,fields:["a","b"]});

Whennotexplicitlyset,thesparseattributedefaultstofalsefornewhashorskiplistindexes.

Thiscausesachangeinbehaviorwhencreatingauniquehashindexwithoutspecifyingthesparseflag:in2.4,uniquehashindexeswereimplicitlysparse,alwaysexcludingnullvalues.Therewasnooptiontocontrolthisbehavior,andsparsitywasneithersupportedfornon-uniquehashindexesnorskiplistsin2.4.Thisimplicitsparsityofuniquehashindexeswasconsideredaninconsistency,andthereforethebehaviorwascleanedupin2.5.Asof2.5,indexeswillonlybecreatedsparseifsparsityisexplicitlyrequested.Existinguniquehashindexesfrom2.4orbeforewillautomaticallybemigratedsotheyarestillsparseaftertheupgradeto2.5.

WhatsNewin2.5

Geoindexesareimplicitlysparse,meaningdocumentswithouttheindexedlocationattributeorcontaininginvalidlocationcoordinatevalueswillbeexcludedfromtheindexautomatically.Thisisalsoachangewhencomparedtopre-2.5behavior,whendocumentswithmissingorinvalidcoordinatevaluesmayhavecausederrorsoninsertionwhenthegeoindex'uniqueflagwassetanditsignoreNullflagwasnot.Thiswasconfusingandhasbeenrectifiedin2.5.ThemethodensureGeoConstraint()nowdoesthesameasensureGeoIndex().Furthermore,theattributesconstraint,unique,ignoreNullandsparseflagsarenowcompletelyignoredwhencreatinggeoindexes.

Thesameistrueforfulltextindexes.Thereisnoneedtospecifynon-uniquenessorsparsityforgeoorfulltextindexes.

Assparseindexesmayexcludesomedocuments,theycannotbeusedforeverytypeofquery.Sparsehashindexescannotbeusedtofinddocumentsforwhichatleastoneoftheindexedattributeshasavalueofnull.Forexample,thefollowingAQLquerycannotuseasparseindex,evenifonewascreatedonattributeattr:

FORdocIncollection

FILTERdoc.attr==null

RETURNdoc

Ifthelookupvalueisnon-constant,asparseindexmayormaynotbeused,dependingontheothertypesofconditionsinthequery.Iftheoptimizercansafelydeterminethatthelookupvaluecannotbenull,asparseindexmaybeused.Whenuncertain,theoptimizerwillnotmakeuseofasparseindexinaqueryinordertoproducecorrectresults.

Forexample,thefollowingqueriescannotuseasparseindexonattrbecausetheoptimizerwillnotknowbeforehandwhetherthecomparisonvaluesfordoc.attrwillincludenull:

FORdocIncollection

FILTERdoc.attr==SOME_FUNCTION(...)

RETURNdoc

FORotherINotherCollection

FORdocIncollection

FILTERdoc.attr==other.attr

RETURNdoc

Sparseskiplistindexescanbeusedforsortingiftheoptimizercansafelydetectthattheindexrangedoesnotincludenullforanyoftheindexattributes.

Selectivityestimates

Indexesoftypeprimary,edgeandhashnowprovideselectivityestimates.ThesewillbeusedbytheAQLqueryoptimizerwhendecidingaboutindexusage.Usingselectivityestimatescanleadtofasterqueryexecutionwhenmoreselectiveindexesareused.

TheselectivityestimatesarealsoreturnedbytheGET/_api/indexRESTAPImethodinasub-attributeselectivityEstimateforeachindexthatsupportsit.Thisattributewillbeomittedforindexesthatdonotprovideselectivityestimates.Ifprovided,theselectivityestimatewillbeanumericvaluebetween0and1.

Selectivityestimateswillalsobereportedintheresultofcollection.getIndexes()forallindexesthatsupportthis.Ifnoselectivityestimatecanbedeterminedforanindex,theattributeselectivityEstimatewillbeomittedhere,too.

Thewebinterfacealsoshowsselectivityestimatesforeachindexthatsupportsthis.

Currentlythefollowingindextypescanprovideselectivityestimates:

primaryindexedgeindexhashindex(uniqueandnon-unique)

Noselectivityestimateswillbeprovidedforindexeswhenrunninginclustermode.

AQLOptimizerimprovements

Sortremoval

WhatsNewin2.5

TheAQLoptimizerrule"use-index-for-sort"willnowremovesortsalsoincaseanon-sortedindex(e.g.ahashindex)isusedforonlyequalitylookupsandallsortattributesarecoveredbytheequalitylookupconditions.

Forexample,inthefollowingquerytheextrasortondoc.valuewillbeoptimizedawayprovidedthereisanindexondoc.value):

FORdocINcollection

FILTERdoc.value==1

SORTdoc.value

RETURNdoc

TheAQLoptimizerrule"use-index-for-sort"nowalsoremovessortincasethesortcriteriaexcludestheleft-mostindexattributes,buttheleft-mostindexattributesareusedbytheindexforequality-onlylookups.

Forexample,inthefollowingquerywithaskiplistindexonvalue1,value2,thesortcanbeoptimizedaway:

FORdocINcollection

FILTERdoc.value1==1

SORTdoc.value2

RETURNdoc

Constantattributepropagation

ThenewAQLoptimizerrulepropagate-constant-attributeswilllookforattributesthatareequality-comparedtoaconstantvalue,andwillpropagatethecomparisonvalueintootherequalitylookups.ThisrulewillonlylookinsideFILTERconditions,andinsertconstantvaluesfoundinFILTERs,too.

Forexample,therulewillinsert42insteadofi.valueinthesecondFILTERofthefollowingquery:

FORiINc1

FORjINc2

FILTERi.value==42

FILTERj.value==i.value

RETURN1

Interleavedprocessing

TheoptimizerwillnowinspectAQLdata-modificationqueriesanddetectifthequery'sdata-modificationpartcanruninlockstepwiththedataretrievalpartofthequery,orifthedataretrievalpartmustbeexecutedandcompletedfirstbeforethedata-modificationcanstart.

Executingbothdataretrievalanddata-modificationinlockstepallowsusingmuchsmallerbuffersforintermediateresults,reducingthememoryusageofqueries.Notallqueriesareeligibleforthisoptimization,andtheoptimizerwillonlyapplytheoptimizationwhenitcansafelydetectthatthedata-modificationpartofthequerywillnotmodifydatatobefoundbytheretrievalpart.

Queryexecutionstatistics

ThefilteredattributewasaddedtoAQLqueryexecutionstatistics.ThevalueofthisattributeindicateshowmanydocumentswerefilteredbyFilterNodesintheAQLquery.NotethatIndexRangeNodescanalsofilterdocumentsbyselectingonlytherequiredrangesfromtheindex.ThefilteredvaluewillnotincludetheworkdonebyIndexRangeNodes,butonlytheworkperformedbyFilterNodes.

Languageimprovements

DynamicattributenamesinAQLobjectliterals

ThischangeallowsusingarbitraryexpressionstoconstructattributenamesinobjectliteralsspecifiedinAQLqueries.Todisambiguateexpressionsandotherunquotedattributenames,dynamicattributenamesneedtobeenclosedinbrackets([and]).

Example:

WhatsNewin2.5

FORiIN1..100

RETURN{[CONCAT('value-of-',i)]:i}

AQLfunctions

ThefollowingAQLfunctionswereaddedin2.5:

MD5(value):generatesanMD5hashofvalueSHA1(value):generatesanSHA1hashofvalueRANDOM_TOKEN(length):generatesarandomstringvalueofthespecifiedlength

SimplifyFoxxusage

ThankstoouruserfeedbackwelearnedthatFoxxisapowerful,yetrathercomplicatedconcept.With2.5wemadeitlesscomplicatedwhilekeepingallitsstrength.Thatincludesarewriteofthedocumentationaswellassomecodechangesasfollows:

MovedFoxxapplicationstoadifferentfolder.

Until2.4foxxappswerestoredinthefollowingfolderstructure:<app-path>/databases/<dbname>/<appname>:<appversion>.Thiscausedsometroubleasappswherecachedbasedonnameandversionandupdatesdidnotapply.Alsothepathonfilesystemandtheapp'saccessURLhadnorelationtooneanother.NowthepathonfilesystemisidenticaltotheURL(excepttheappendedAPP):<app-path>/_db/<dbname>/<mointpoint>/APP

RewriteofFoxxrouting

TheroutingofFoxxhasbeenexposedtomajorinternalchangesweadjustedbecauseofuserfeedback.Thisallowsustosetthedevelopmentmodepermountpointwithouthavingtochangepathsandholdappsatseparatelocations.

FoxxDevelopmentmode

Thedevelopmentmodeuseduntil2.4isgone.Ithasbeenreplacedbyamuchmorematureversion.Thisincludesthedeprecationofthejavascript.dev-app-pathparameter,whichisuselesssince2.5.Insteadofhavingtwoseparateappdirectoriesforproductionanddevelopment,appsnowresideinoneplace,whichisusedforproductionaswellasfordevelopment.Appscanstillbeputintodevelopmentmode,changingtheirbehaviorcomparedtoproductionmode.Developmentmodeappsarestillrereadfromdiskateveryrequest,andstilltheyshipmoredebugoutput.

Thischangehasalsomadethestartupoptions--javascript.frontend-development-modeand--javascript.dev-app-pathobsolete.Theformeroptionwillnothaveanyeffectwhenset,andthelatteroptionisonlyreadandusedduringtheupgradeto2.5anddoesnothaveanyeffectslater.

Foxxinstallprocess

InstallingFoxxappshasbeenatwostepprocess:importthemintoArangoDBandmountthemataspecificmountpoint.Theseoperationshavebeenjoinedtogether.Youcaninstallanappatonemountpoint,that'sit.Nofetch,mount,unmount,purgecycleanymore.Thecommandshavebeensimplifiedtojust:

install:getyourFoxxappupandrunninguninstall:shutitdownanderaseitfromdisk

Foxxerroroutput

Until2.4theerrorsproducedbyFoxxwerenotoptimal.Often,theerrormessagewasjustunabletoparsemanifestandcontainedonlyaninternalstacktrace.In2.5wemademajorimprovementsthere,includingamuchmorefinegrainederroroutputthathelpsyoudebugyourFoxxapps.Theerrormessageprintedisnowmuchclosertoitssourceandshouldhelpyoutrackitdown.

AlsoweaddedthedefaulthandlersforunhandlederrorsinFoxxapps:

YouwillgetaniceinternalerrorpagewheneveryourFoxxappiscalledbutwasnotinstalledduetoanyerror

WhatsNewin2.5

Youwillgetapropererrormessagewhenhavinganuncaughterrorappearsinanyapproute

InproductionmodethemessagesabovewillNOTcontainanyinformationaboutyourFoxxinternalsandaresafetobeexposedtothirdpartyusers.Indevelopmentmodethemessagesabovewillcontainthestacktrace(ifavailable),makingiteasierforyourin-housedevstotrackdownerrorsintheapplication.

Foxxconsole

WeaddedaconsoleobjecttoFoxxapps.AllFoxxappsnowhaveaconsoleobjectimplementingthefamiliarConsoleAPIintheirglobalscope,whichcanbeusedtologdiagnosticmessagestothedatabase.Thisconsolealsoallowstoreadtheerroroutputofonespecificfoxx.

Foxxrequests

Weaddedorg/arangodb/requestmodule,whichprovidesasimpleAPIformakingHTTPrequeststoexternalservices.ThisisenablesFoxxtobedirectlypartofamicroservicearchitecture.

WhatsNewin2.5

Changedbehavior

TheV8versionshippedwithArangoDBwasupgradedfrom3.29.59to3.31.74.1.ThisleadstoadditionalECMAScript6(ES6or"harmony")featuresbeingenabledbydefaultinArangoDB'sscriptingenvironment.

Apartfromthat,achangeintheinterpretationofcommand-lineoptionsbyV8mayaffectusers.ArangoDBpassesthevalueofthecommand-lineoption--javascript.v8-optionstoV8andleavesinterpretationofthecontentstoV8.Forexample,theArangoDBoption--javascript.v8-options="--harmony"couldbeusedtotellV8toenableitsharmonyfeatures.

InArangoDB2.4,thefollowingharmonyoptionsweremadeavailablebyV8:

--harmony_scoping(enableharmonyblockscoping)--harmony_modules(enableharmonymodules(impliesblockscoping))--harmony_proxies(enableharmonyproxies)--harmony_generators(enableharmonygenerators)--harmony_numeric_literals(enableharmonynumericliterals(0o77,0b11))--harmony_strings(enableharmonystring)--harmony_arrays(enableharmonyarrays)--harmony_arrow_functions(enableharmonyarrowfunctions)--harmony_classes(enableharmonyclasses)--harmony_object_literals(enableharmonyobjectliteralextensions)--harmony(enableallharmonyfeatures(exceptproxies))

Therewastheoption--harmony,whichturnedonalmostallharmonyfeatures.

InArangoDB2.5,V8providesthefollowingharmony-relatedoptions:

--harmony(enableallcompletedharmonyfeatures)--harmony_shipping(enableallshippedharmonyfeatures)--harmony_modules(enable"harmonymodules(impliesblockscoping)"(inprogress))--harmony_arrays(enable"harmonyarraymethods"(inprogress))--harmony_array_includes(enable"harmonyArray.prototype.includes"(inprogress))--harmony_regexps(enable"harmonyregularexpressionextensions"(inprogress))--harmony_arrow_functions(enable"harmonyarrowfunctions"(inprogress))--harmony_proxies(enable"harmonyproxies"(inprogress))--harmony_sloppy(enable"harmonyfeaturesinsloppymode"(inprogress))--harmony_unicode(enable"harmonyunicodeescapes"(inprogress))--harmony_tostring(enable"harmonytoString")--harmony_numeric_literals(enable"harmonynumericliterals")--harmony_strings(enable"harmonystringmethods")--harmony_scoping(enable"harmonyblockscoping")--harmony_classes(enable"harmonyclasses(impliesblockscoping&objectliteralextension)")--harmony_object_literals(enable"harmonyobjectliteralextensions")--harmony_templates(enable"harmonytemplateliterals")

Notethatthereareextraoptionsforbettercontrollingthededicatedfeatures,andespeciallythatthemeaningofthe--harmonyoptionhaschangedfromenablingallharmonyfeaturestoallcompletedharmonyfeatures!

Usersshouldadjustthevalueof--javascript.v8-optionsaccordingly.

PleasenotethatincompleteharmonyfeaturesaresubjecttochangeinfutureV8releases.

Sparseindexes

Hashindexesandskiplistindexescannowbecreatedinasparsevariant.Whennotexplicitlyset,thesparseattributedefaultstofalsefornewindexes.

Thiscausesachangeinbehaviorwhencreatingauniquehashindexwithoutspecifyingthesparseflag.Theuniquehashindexwillbecreatedinanon-sparsevariantinArangoDB2.5.

In2.4andbefore,uniquehashindexeswereimplicitlysparse,alwaysexcludingnullvaluesfromtheindex.Therewasnooptiontocontrolthisbehavior,andsparsitywasneithersupportedfornon-uniquehashindexesnorskiplistsin2.4.Thisimplicitsparsityofjustuniquehashindexeswasconsideredaninconsistency,andthereforethebehaviorwascleanedupin2.5.

Asof2.5,hashandskiplistindexeswillonlybecreatedsparseifsparsityisexplicitlyrequested.Thismayrequireachangeinindex-creatingclientcode,butonlyiftheclientcodecreatesuniquehashindexesandiftheyarestillintendedtobesparse.Inthiscase,theclientcodeshouldexplicitlysetthesparseflagtotruewhencreatingauniquehashindex.

Existinguniquehashindexesfrom2.4orbeforewillautomaticallybemigratedsotheyarestillsparseaftertheupgradeto2.5.Fortheseindexes,thesparseattributewillbepopulatedautomaticallywithavalueoftrue.

Geoindexesareimplicitlysparse,meaningdocumentswithouttheindexedlocationattributeorcontaininginvalidlocationcoordinatevalueswillbeexcludedfromtheindexautomatically.Thisisalsoachangewhencomparedtopre-2.5behavior,whendocumentswithmissingorinvalidcoordinatevaluesmayhavecausederrorsoninsertionwhenthegeoindex'uniqueflagwassetanditsignoreNullflagwasnot.

Thiswasconfusingandhasbeenrectifiedin2.5.ThemethodensureGeoConstraint()nowdoesthesameasensureGeoIndex().Furthermore,theattributesconstraint,unique,ignoreNullandsparseflagsarenowcompletelyignoredwhencreatinggeoindexes.ClientindexcreationcodethereforedoesnotneedtosettheignoreNullorconstraintattributeswhencreatingageoindex.

Thesameistrueforfulltextindexes.Thereisnoneedtospecifynon-uniquenessorsparsityforgeoorfulltextindexes.Theywillalwaysbenon-uniqueandsparse.

MovedFoxxapplicationstoadifferentfolder.

Until2.4foxxappswerestoredinthefollowingfolderstructure:<app-path>/databases/<dbname>/<appname>:<appversion>.Thiscausedsometroubleasappswherecachedbasedonnameandversionandupdatesdidnotapply.Alsothepathonfilesystemandtheapp'saccessURLhadnorelationtooneanother.NowthepathonfilesystemisidenticaltotheURL(excepttheappendedAPP):<app-path>/_db/<dbname>/<mointpoint>/APP

FoxxDevelopmentmode

Thedevelopmentmodeuseduntil2.4isgone.Ithasbeenreplacedbyamuchmorematureversion.Thisincludesthedeprecationofthejavascript.dev-app-pathparameter,whichisuselesssince2.5.Insteadofhavingtwoseparateappdirectoriesforproductionanddevelopment,appsnowresideinoneplace,whichisusedforproductionaswellasfordevelopment.Appscanstillbeputintodevelopmentmode,changingtheirbehaviorcomparedtoproductionmode.Developmentmodeappsarestillrereadfromdiskateveryrequest,andstilltheyshipmoredebugoutput.

Thischangehasalsomadethestartupoptions--javascript.frontend-development-modeand--javascript.dev-app-pathobsolete.Theformeroptionwillnothaveanyeffectwhenset,andthelatteroptionisonlyreadandusedduringtheupgradeto2.5anddoesnothaveanyeffectslater.

Foxxinstallprocess

InstallingFoxxappshasbeenatwostepprocess:importthemintoArangoDBandmountthemataspecificmountpoint.Theseoperationshavebeenjoinedtogether.Youcaninstallanappatonemountpoint,that'sit.Nofetch,mount,unmount,purgecycleanymore.Thecommandshavebeensimplifiedtojust:

install:getyourFoxxappupandrunninguninstall:shutitdownanderaseitfromdiskDeprecatedfeatures

Foxx:methodModel#toJSONSchema(id)isdeprecated,itwillraiseawarningifyouuseit.PleaseuseFoxx.toJSONSchema(id,model)

instead.

Removedfeatures

Startupswitch--javascript.frontend-development-mode:Itsmajorpurposewasinternaldevelopmentanyway.Nowthewebfrontendcanbesettodevelopmentmodesimilartoanyotherfoxxapp.Startupswitch--javascript.dev-app-path:WasusedforthedevelopmentmodeofFoxx.Thisisintegratedwiththenormalapp-pathnowandcanbetriggeredonapplevel.Thesecondapp-pathissuperfluous.Foxx:controller.collection:PleaseuseappContext.collectioninstead.Foxx:FoxxRepository.modelPrototype:PleaseuseFoxxRepository.modelinstead.Foxx:Model.extend({},{attributes:{}}):PleaseuseModel.extend({schema:{}})instead.Foxx:requestContext.bodyParam(paramName,description,Model):PleaseuserequestContext.bodyParam(paramName,options)instead.Foxx:requestContext.queryParam({type:string}):PleaseuserequestContext.queryParam({type:joi})instead.Foxx:requestContext.pathParam({type:string}):PleaseuserequestContext.pathParam({type:joi})instead.Graph:Themodulesorg/arangodb/graphandorg/arangodb/graph-blueprint:Pleaseusemoduleorg/arangodb/general-graphinstead.NOTE:Thisdoesnotmeanwedonotsupportblueprintsanymore.Generalgraphcoverseverythingthegraph--blueprintdid,plusmanymorefeatures.General-Graph:Inthemoduleorg/arangodb/general-graphthefunctions_undirectedRelationand_directedRelationarenolongeravailable.Bothfunctionshavebeenunifiedto_relation.

V8versionupgrade

Thebuilt-inversionofV8hasbeenupgradedfrom3.16.14to3.29.59.ThisactivatesseveralES6(alsodubbedHarmonyorES.next)featuresinArangoDB,bothintheArangoShellandtheArangoDBserver.Theycanbeusedforscriptingandinserver-sideactionssuchasFoxxroutes,traversalsetc.

ThefollowingES6featuresareavailableinArangoDB2.4bydefault:

iteratorstheofoperatorsymbolspredefinedcollectionstypes(Map,Setetc.)typedarrays

ManyotherES6featuresaredisabledbydefault,butcanbemadeavailablebystartingarangodorarangoshwiththeappropriateoptions:

arrowfunctionsproxiesgeneratorsString,Array,andNumberenhancementsconstantsenhancedobjectandnumericliterals

ToactivatealltheseES6featuresinarangodorarangosh,startitwiththefollowingoptions:

arangosh--javascript.v8-options="--harmony--harmony_generators"

MoredetailsontheavailableES6featurescanbefoundinthisblog.

FoxxGeneratorArangoDB2.4isshippedwithFoxxGenerator,aframeworkforbuildingstandardizedHypermediaAPIseasily.ThegeneratedAPIscanbeconsumedwithclienttoolsthatunderstandSiren.

HypermediaisthesimpleideathatourHTTPAPIsshouldhavelinksbetweentheirendpointsinthesamewaythatourwebsiteshavelinksbetweenthem.FoxxGeneratorisbasedontheideathatyoucanrepresentanAPIasastatechart:Everyendpointisastateandthelinksarethetransitionsbetweenthem.Usingyourdescriptionofstatesandtransitions,itcanthencreateanAPIforyou.

TheFoxxGeneratorcancreateAPIsbasedonasemanticdescriptionofentitiesandtransitions.AblogseriesontheusecasesandhowtousetheFoxxgeneratorishere:

part1part2part3

AcookbookrecipeforgettingstartedwithFoxxGeneratorishere.

AQLimprovements

WhatsNewin2.4

TheAQLoptimizerhasbeenenhancedtouseofindexesinqueriesinseveraladditionalcases.FilterscontainingtheINoperatorcannowmakeuseofindexes,andmultipleOR-orAND-combinedfilterconditionscannowalsouseindexesifthefilterconditionsrefertothesameindexedattribute.

Hereareafewexamplesofqueriesthatcannowuseindexesbutcouldn'tbefore:

FORdocINcollection

FILTERdoc.indexedAttribute==1||doc.indexedAttribute>99

RETURNdoc

FORdocINcollection

FILTERdoc.indexedAttributeIN[3,42]||doc.indexedAttribute>99

RETURNdoc

FORdocINcollection

FILTER(doc.indexedAttribute>2&&doc.indexedAttribute<10)||

(doc.indexedAttribute>23&&doc.indexedAttribute<42)

RETURNdoc

Additionally,theoptimizerruleremove-filter-covered-by-indexhasbeenadded.ThisruleremovesFilterNodesandCalculationNodesfromanexecutionplanifthefilterconditionisalreadycoveredbyapreviousIndexRangeNode.Removingthefilter'sCalculationNodeandtheFilterNodeitselfwillspeedupqueryexecutionbecausethequeryrequireslesscomputation.

Furthermore,thenewoptimizerruleremove-sort-randwillremoveaSORTRAND()statementandmovetherandomiterationintotheappropriateEnumerateCollectionNode.Thisisusuallymoreefficientthanindividuallyenumeratingandsorting.

Data-modificationqueriesreturningdocuments

INSERT,REMOVE,UPDATEorREPLACEqueriesnowcanoptionallyreturnthedocumentsinserted,removed,updated,orreplaced.Thisishelpfulfortrackingtheauto-generatedattributes(e.g._key,_rev)createdbyanINSERTandinalotofothersituations.

Inordertoreturndocumentsfromadata-modificationquery,thestatementmustimmediatelybeimmediatelyfollowedbyaLETstatementthatassignseitherthepseudo-valueNEWorOLDtoavariable.ThisLETstatementmustbefollowedbyaRETURNstatementthatreturnsthevariableintroducedbyLET:

FORiIN1..100

INSERT{value:i}INtestLETinserted=NEWRETURNinserted

FORuINusers

FILTERu.status=='deleted'

REMOVEuINusersLETremoved=OLDRETURNremoved

FORuINusers

FILTERu.status=='notactive'

UPDATEuWITH{status:'inactive'}INusersLETupdated=NEWRETURNupdated

NEWreferstotheinsertedormodifieddocumentrevision,andOLDreferstothedocumentrevisionbeforeupdateorremoval.INSERTstatementscanonlyrefertotheNEWpseudo-value,andREMOVEoperationsonlytoOLD.UPDATEandREPLACEcanrefertoeither.

Inallcasesthefulldocumentswillbereturnedwithalltheirattributes,includingthepotentiallyauto-generatedattributessuchas_id,_key,or_revandtheattributesnotspecifiedintheupdateexpressionofapartialupdate.

COUNTclause

AnoptionalCOUNTclausewasaddedtotheCOLLECTstatement.TheCOUNTclauseallowsformoreefficientcountingofvalues.

InpreviousversionsofArangoDBonehadtowritethefollowingtocountdocuments:

RETURNLENGTH(

FORdocINcollection

FILTER...somecondition...

RETURNdoc

WhatsNewin2.4

WiththeCOUNTclause,thequerycanbemodifiedto

FORdocINcollection

COLLECTWITHCOUNTINTOlength

RETURNlength

ThelatterquerywillbemuchmoreefficientbecauseitwillnotproduceanyintermediateresultswithneedtobeshippedfromasubqueryintotheLENGTHfunction.

TheCOUNTclausecanalsobeusedtocountthenumberofitemsineachgroup:

FORdocINcollection

COLLECTgroup=doc.groupWITHCOUNTINTOlength

return{group:group,length:length}

COLLECTmodifications

InArangoDB2.4,COLLECToperationscanbemademoreefficientifonlyasmallfragmentofthegroupvaluesisneededlater.Forthesecases,COLLECTprovidesanoptionalconversionexpressionfortheINTOclause.Thisexpressioncontrolsthevaluethatisinsertedintothearrayofgroupvalues.Itcanbeusedforprojections.

ThefollowingqueryonlycopiesthedateRegisteredattributeofeachdocumentintothegroups,potentiallysavingalotofmemoryandcomputationtimecomparedtocopyingdoccompletely:

FORdocINcollection

COLLECTgroup=doc.groupINTOdates=doc.dateRegistered

return{group:group,maxDate:MAX(dates)}

Comparethistothefollowingvariantofthequery,whichwastheonlywaytoachievethesameresultinpreviousversionsofArangoDB:

FORdocINcollection

COLLECTgroup=doc.groupINTOdates

return{group:group,maxDate:MAX(dates[*].doc.dateRegistered)}

Theabovequerywillneedtocopythefulldocattributeintothelengthsvariable,whereasthenewvariantwillonlycopythedateRegisteredattributeofeachdoc.

Subquerysyntax

InpreviousversionsofArangoDB,subqueriesrequiredextraparenthesesaroundthem,andthiscausedconfusionwhensubquerieswereusedasfunctionparameters.Forexample,thefollowingquerydidnotwork:

LETvalues=LENGTH(

FORdocINcollectionRETURNdoc

buthadtobewrittenasfollows:

LETvalues=LENGTH((

FORdocINcollectionRETURNdoc

Thiswasunintuitiveandisfixedinversion2.4sothatbothvariantsofthequeryareacceptedandproducethesameresult.

Webinterface

WhatsNewin2.4

TheApplicationstabforFoxxapplicationsinthewebinterfacehasgotacompleteredesign.

ItwillnowonlyshowapplicationsthatarecurrentlyrunninginArangoDB.Foraselectedapplication,anewdetailedviewhasbeencreated.Thisviewprovidesabetteroverviewoftheapp,e.g.:

authorlicenseversioncontributorsdownloadlinksAPIdocumentation

InstallinganewFoxxapplicationontheserverismadeeasyusingthenewAddapplicationbutton.TheAddapplicationdialogprovidesallthefeaturesalreadyavailableinthefoxx-managerconsoleapplicationplussomemore:

installaFoxxapplicationfromGithubinstallaFoxxapplicationfromazipfileinstallaFoxxapplicationfromArangoDB'sapplicationstorecreateanewFoxxapplicationfromscratch:thisfeatureusesageneratortocreateaFoxxapplicationwithpre-definedCRUDmethodsforagivenlistofcollections.ThegeneratedFoxxappcaneitherbedownloadedasazipfileorbeinstalledontheserver.StartingwithanewFoxxapphasneverbeeneasier.

Defaultendpointis127.0.0.1

ThedefaultendpointfortheArangoDBserverhasbeenchangedfrom0.0.0.0to127.0.0.1.ThiswillmakenewArangoDBinstallationsunaccessiblefromclientsotherthanlocalhostunlesstheconfigurationischanged.Thisisasecurityprecautionmeasurethathasbeenrequestedasafeaturealotoftimes.

Ifyouarethedevelopmentoption--enable-relative,theendpointwillstillbe0.0.0.0.

Systemcollectionsinreplication

Bydefault,systemcollectionsarenowincludedinreplicationandallreplicationAPIreturnvalues.Thiswillleadtouseraccountsandcredentialsdatabeingreplicatedfrommastertoslaveservers.Thismayoverwriteslave-specificdatabaseusers.

Ifthisisundesired,the_userscollectioncanbeexcludedfromreplicationeasilybysettingtheincludeSystemattributetofalseinthefollowingcommands:

replication.sync({includeSystem:false});replication.applier.properties({includeSystem:false});

Thiswillexcludeallsystemcollections(including_aqlfunctions,_graphsetc.)fromtheinitialsynchronizationandthecontinuousreplication.

Ifthisisalsoundesired,itisalsopossibletospecifyalistofcollectionstoexcludefromtheinitialsynchronizationandthecontinuousreplicationusingtherestrictCollectionsattribute,e.g.:

require("org/arangodb/replication").applier.properties({

includeSystem:true,

restrictType:"exclude",

restrictCollections:["_users","_graphs","foo"]

WhatsNewin2.4

Changedbehavior

V8upgrade

ThebundledV8versionhasbeenupgradedfrom3.16.14to3.29.59.

Thenewversionprovidesbettererrorchecking,whichcanleadtosubtlechangesintheexecutionofJavaScriptcode.

Thefollowingcode,thoughnonsense,runswithouterrorin2.3and2.4whenstrictmodeisnotenabled:

(function(){

a=true;

a.foo=1;

Whenenablingstrictmode,thefunctionwillthrowanerrorin2.4butnotin2.3:

(function(){

"usestrict";

a=true;

a.foo=1;

TypeError:Cannotassigntoreadonlyproperty'foo'oftrue

Thoughthisisachangeinbehavioritcanbeconsideredanimprovement.Thenewversionactuallyuncoversanerrorthatwentundetectedintheoldversion.

Errormessageshavealsochangedslightlyinthenewversion.ApplicationsthatrelyontheexacterrormessagesoftheJavaScriptenginemayneedtobeadjustedsotheylookfortheupdatederrormessages.

Defaultendpoint

Thedefaultendpointforarangodisnow127.0.0.1.

ThischangewillmodifytheIPaddressArangoDBlistensonto127.0.0.1bydefault.ThiswillmakenewArangoDBinstallationsunaccessiblefromclientsotherthanlocalhostunlesstheconfigurationischanged.Thisisasecurityfeature.

TomakeArangoDBaccessiblefromanyclient,changetheserver'sconfiguration(--server.endpoint)toeithertcp://0.0.0.0:8529ortheserver'spubliclyvisibleIPaddress.

Replication

SystemcollectionsarenowincludedinthereplicationandallreplicationAPIreturnvaluesbydefault.

Thiswillleadtouseraccountsandcredentialsdatabeingreplicatedfrommastertoslaveservers.Thismayoverwriteslave-specificdatabaseusers.

Thismaybeconsideredafeatureorananti-feature,soitisconfigurable.

Ifreplicationofsystemcollectionsisundesired,theycanbeexcludedfromreplicationbysettingtheincludeSystemattributetofalseinthefollowingcommands:

initialsynchronization:replication.sync({includeSystem:false})continuousreplication:replication.applier.properties({includeSystem:false})

Thiswillexcludeallsystemcollections(including_aqlfunctions,_graphsetc.)fromtheinitialsynchronizationandthecontinuousreplication.

Ifthisisalsoundesired,itisalsopossibletospecifyalistofcollectionstoexcludefromtheinitialsynchronizationandthecontinuousreplicationusingtherestrictCollectionsattribute,e.g.:

require("org/arangodb/replication").applier.properties({

includeSystem:true,

restrictType:"exclude",

restrictCollections:["_users","_graphs","foo"]

Theaboveexamplewillingeneralincludesystemcollections,butwillexcludethespecifiedthreecollectionsfromcontinuousreplication.

TheHTTPRESTAPImethodsforfetchingthereplicationinventoryandfordumpingcollectionsalsosupporttheincludeSystemcontrolflagviaaURLparameterofthesamename.

Buildprocesschanges

Severaloptionsfortheconfigurecommandhavebeenremovedin2.4.Theoptions

--enable-all-in-one-v8

--enable-all-in-one-icu

--enable-all-in-one-libev

--with-libev=DIR

--with-libev-lib=DIR

--with-v8=DIR

--with-v8-lib=DIR

--with-icu-config=FILE

arenotavailableanymorebecausethebuildprocesswillalwaysusethebundledversionsofthelibraries.

WhenbuildingArangoDBfromsourceinadirectorythatalreadycontainedapre-2.4version,itwillbenecessarytorunamakesupercleancommandonceandafullrebuildafterwards:

gitpull

makesuperclean

makesetup

./configure<optionsgohere>

MiscellaneouschangesAsaconsequenceofglobalrenaminginthecodebase,theoptionmergeArrayshasbeenrenamedtomergeObjects.ThisoptioncontrolswhetherJSONobjectswillbemergedonanupdateoperationoroverwritten.Thedefaulthasbeen,andstillis,tomerge.Notspecifyingtheparameterwillleadtoamerge,asithasbeenthebehaviorinArangoDBeversince.

ThisaffectstheHTTPRESTAPImethodPATCH/_api/document/collection/key.ItsoptionalURLparametermergeArraysfortheoptionhasbeenrenamedtomergeObjects.

TheAQLUPDATEstatementisalsoaffected,asitsoptionmergeArrayshasalsobeenrenamedtomergeObjects.The2.3query

UPDATEdocINcollectionWITH{...}INcollectionOPTIONS{mergeArrays:false}

shouldthusberewrittentothefollowingin2.4:

UPDATEdocINcollectionWITH{...}INcollectionOPTIONS{mergeObjects:false}

Deprecatedfeatures

ForFoxxControllerobjects,themethodcollection()isdeprecatedandwillberemovedinfutureversionofArangoDB.Usingthismethodwillissueawarning.PleaseuseapplicationContext.collection()instead.

ForFoxxRepositoryobjects,thepropertymodelPrototypeisnowdeprecated.Usingitwillissueawarning.PleaseuseFoxxRepository.modelinstead.

InFoxxController/RequestContext,callingmethodbodyParam()withthreeargumentsisdeprecated.Pleaseuse.bodyParam(paramName,options)instead.

InFoxxController/RequestContextcallingmethodqueryParam({type:string})isdeprecated.PleaseuserequestContext.queryParam({type:joi})instead.

InFoxxController/RequestContextcallingmethodpathParam({type:string})isdeprecated.PleaseuserequestContext.pathParam({type:joi})instead.

ForFoxxModel,callingModel.extend({},{attributes:{}})isdeprecated.PleaseuseModel.extend({schema:{}})instead.

Inmoduleorg/arangodb/general-graph,thefunctions_undirectedRelation()and_directedRelation()aredeprecatedandwillberemovedinafutureversionofArangoDB.Bothfunctionshavebeenunifiedto_relation().

Themodulesorg/arangodb/graphandorg/arangodb/graph-blueprintaredeprecated.Pleaseusemoduleorg/arangodb/general-graphinstead.

TheHTTPRESTAPI_api/graphandallitsmethodsaredeprecated.PleaseusethegeneralgraphAPI_api/gharialinstead.

Removedfeatures

Thefollowingreplication-relatedJavaScriptmethodsbecameobsoleteinArangoDB2.2andhavebeenremovedinArangoDB2.4:

require("org/arangodb/replication").logger.start()

require("org/arangodb/replication").logger.stop()

require("org/arangodb/replication").logger.properties()

TheRESTAPImethodsforthesefunctionshavealsobeenremovedinArangoDB2.4:

HTTPPUT/_api/replication/logger-startHTTPPUT/_api/replication/logger-stopHTTPGET/_api/replication/logger-configHTTPPUT/_api/replication/logger-config

Clientapplicationsthatcalloneofthesemethodsshouldbeadjustedbyremovingthecallstothesemethods.Thisshouldn'tbeproblematicasthesemethodshavebeenno-opssinceArangoDB2.2anyway.

AQLimprovements

Frameworkimprovements

AQLqueriesarenowsentthroughaqueryoptimizerframeworkbeforeexecution.Thequeryoptimizerframeworkwillfirstconverttheinternalrepresentationofthequery,theabstractsyntaxtree,intoaninitialexecutionplan.

Theexecutionplanisthensendthroughoptimizerrulesthatmaydirectlymodifytheplaninplaceorcreateanewvariantoftheplan.Newplansmightagainbeoptimized,allowingtheoptimizertocarryoutseveraloptimizations.

Aftercreatingplans,theoptimizerwillestimatethecostsforeachplanandpicktheplanwiththelowestcost(termedtheoptimalplan)fortheactualqueryexecution.

Withtheexplain()methodofArangoStatementuserscancheckwhichexecutionplantheoptimizerpickorretrievealistofotherplansthatoptimizerdidnotchoose.Theplanwillrevealmanydetailsaboutwhichindexesareusedetc.explain()willalsoreturntheofoptimizerrulesappliedsouserscanvalidatewhetherornotaqueryallowsusingaspecificoptimization.

ExecutionofAQLquerieshasbeenrewritteninC++,allowingmanyqueriestoavoidtheconversionofdocumentsbetweenArangoDB'sinternallow-leveldatastructureandtheV8objectrepresentationformat.

Theframeworkforoptimizerrulesisnowalsogenerallycluster-aware,allowingspecificoptimizationsforqueriesthatruninacluster.Additionally,theoptimizerwasdesignedtobeextensibleinordertoaddmoreoptimizationsinthefuture.

Alternativeoperatorsyntax

ArangoDB2.3allowstousethefollowingalternativeformsforthelogicaloperators:

AND:logicalandOR:logicalorNOT:negation

Thisnewsyntaxisjustanalternativetotheoldsyntax,allowingeasiermigrationfromSQL.Theoldsyntaxisstillfullysupportedandwillbe:

&&:logicaland||:logicalor!:negation

NOTINoperator

AQLnowhasadedicatedNOTINoperator.

Previously,aNOTINwasonlyachievablebywritinganegatedINcondition:

FORiIN...FILTER!(iIN[23,42])...

InArangoDB2.3,thesameresultcannowalternativelybeachievedbywritingthemoreintuitivevariant:

FORiIN...FILTERiNOTIN[23,42]...

Improvementsofbuilt-infunctions

WhatsNewin2.3

ThefollowingAQLstringfunctionshavebeenadded:

LTRIM(value,characters):left-trimsastringvalueRTRIM(value,characters):right-trimsastringvalueFIND_FIRST(value,search,start,end):findsthefirstoccurrenceofasearchstringFIND_LAST(value,search,start,end):findsthelastoccurrenceofasearchstringSPLIT(value,separator,limit):splitsastringintoanarray,usingaseparatorSUBSTITUTE(value,search,replace,limit):replacescharactersorstringsinsideanother

ThefollowingotherAQLfunctionshavebeenadded:

VALUES(document):returnsthevaluesofanobjectasanarray(thisisthecounterparttothealreadyexistingATTRIBUTESfunction)ZIP(attributes,values):returnsanobjectconstructedfromattributesandvaluespassedinseparateparametersPERCENTILE(values,n,method):returnsthenthspercentileofthevaluesprovided,usingrankorinterpolationmethod

ThealreadyexistingfunctionsCONCATandCONCAT_SEPARATORnowsupportarrayarguments,e.g.:

/*"foobarbaz"*/

CONCAT(['foo','bar','baz'])

/*"foo,bar,baz"*/

CONCAT_SEPARATOR(",",['foo','bar','baz'])

AQLqueriesthrowlessexceptions

InpreviousversionsofArangoDB,AQLqueriesabortedwithanexceptioninmanysituationsandthrewaruntimeexception.Forexample,exceptionswerethrownwhentryingtofindavalueusingtheINoperatorinanon-arrayelement,whentryingtousenon-booleanvalueswiththelogicaloperands&&or||or!,whenusingnon-numericvaluesinarithmeticoperations,whenpassingwrongparametersintofunctionsetc.

ThefactthatmanyAQLoperatorscouldthrowexceptionsledtoalotofquestionsfromusers,andalotofmore-verbose-than-necessaryqueries.Forexample,thefollowingqueryfailedwhenthereweredocumentsthatdidnothaveatopicsattributeatall:

FORdocINmycollection

FILTERdoc.topicsIN["something","whatever"]

RETURNdoc

Thisforceduserstorewritethequeryasfollows:

FILTERIS_LIST(doc.topics)&&doc.topicsIN["something","whatever"]

RETURNdoc

InArangoDB2.3thishasbeenchangedtomakeAQLeasiertouse.Thechangeprovidesanextrabenefit,andthatisthatnon-throwingoperatorsallowthequeryoptimizertoperformmuchmoretransformationsinthequerywithoutchangingitsoverallresult.

Hereisasummaryofchanges:

whenanon-arrayvalueisusedontheright-handsideoftheINoperator,theresultwillbefalseinArangoDB2.3,andnoexceptionwillbethrown.thebooleanoperators&&and||donotthrowinArangoDB2.3ifanyoftheoperandsisnotabooleanvalue.Instead,theywillperformanimplicitcastofthevaluestobooleans.Theirresultwillbeasfollows:

lhs&&rhswillreturnlhsifitisfalseorwouldbefalsewhenconvertedintoaboolean.Iflhsistrueorwouldbetruewhenconvertedtoaboolean,rhswillbereturned.lhs||rhswillreturnlhsifitistrueorwouldbetruewhenconvertedintoaboolean.Iflhsisfalseorwouldbefalsewhenconvertedtoaboolean,rhswillbereturned.!valuewillreturnthenegatedvalueofvalueconvertedintoaboolean

thearithmeticoperators(+,-,*,/,%)canbeappliedtoanyvalueandwillnotthrowexceptionswhenappliedtonon-numericvalues.Instead,anyvalueusedintheseoperatorswillbecastedtoanumericvalueimplicitly.Ifnonumericresultcanbeproducedbyanarithmeticoperator,itwillreturnnullinArangoDB2.3.Thisisalsotruefordivisionbyzero.passingargumentsofinvalidtypesintoAQLfunctionsdoesnotthrowaruntimeexceptioninmostcases,butmayproduceruntime

WhatsNewin2.3

warnings.Built-inAQLfunctionsthatreceiveinvalidargumentswillthenreturnnull.

Performanceimprovements

Non-uniquehashindexes

Theperformanceofinsertionintonon-uniquehashindexeshasbeenimprovedsignificantly.Thisfixesperformanceproblemsincaseattributeswereindexesthatcontainedonlyveryfewdistinctvalues,orwhenmostofthedocumentsdidnotevencontaintheindexedattribute.Thisalsofixesproblemswhenloadingcollectionswithsuchindexes.

Theinsertiontimenowscaleslinearlywiththenumberofdocumentsregardlessofthecardinalityoftheindexedattribute.

Reverseiterationoverskiplistindexes

AQLqueriescannowuseasortedskiplistindexforreverseiteration.ThisallowsseveralqueriestorunfasterthaninpreviousversionsofArangoDB.

Forexample,thefollowingAQLquerycannowusetheindexondoc.value:

FILTERdoc.value>23

SORTdoc.valuesDESC

RETURNdoc

PreviousversionsofArangoDBdidnotusetheindexbecauseofthedescending(DESC)sort.

Additionally,thenewAQLoptimizercanuseanindexforsortingnoweveniftheAQLquerydoesnotcontainaFILTERstatement.ThisoptimizationwasnotavailableinpreviousversionsofArangoDB.

AddedbasicsupportforhandlingbinarydatainFoxx

BufferobjectscannowbeusedwhensettingtheresponsebodyofanyFoxxaction.ThisallowsFoxxactionstoreturnbinarydata.

RequestswithbinarypayloadcanbeprocessedinFoxxapplicationsbyusingthenewmethodres.rawBodyBuffer().ThiswillreturntheunparsedrequestbodyasaBufferobject.

Thereisnowalsothemethodreq.requestParts()availableinFoxxtoretrievetheindividualcomponentsofamultipartHTTPrequest.Thatcanbeusedforexampletoprocessfileuploads.

Additionally,theres.send()methodhasbeenaddedasaconveniencemethodforreturningstrings,JSONobjectsorBuffersfromaFoxxaction.Itprovidessomeauto-detectionbasedonitsparametervalue:

res.send("<p>someHTML</p>");//returnsanHTMLstring

res.send({success:true});//returnsaJSONobject

res.send(newBuffer("somebinarydata"));//returnsbinarydata

Theconveniencemethodres.sendFile()cannowbeusedtoreturnthecontentsofafilefromaFoxxaction.Theyfilemaycontainbinarydata:

res.sendFile(applicationContext.foxxFilename("image.png"));

Thefilesystemmethodsfs.write()andfs.readBuffer()canbeusedtoworkwithbinarydata,too:

fs.write()willperformanauto-detectionofitssecondparameter'svaluesoitworkswithBufferobjects:

fs.write(filename,"somedata");//savesastringvalueinfile

fs.write(filename,newBuffer("somebinarydata"));//saves(binary)contentsofabuffer

fs.readBuffer()hasbeenaddedasamethodtoreadthecontentsofanarbitraryfileintoaBufferobject.

WhatsNewin2.3

Webinterface

Batchdocumentremovalandmovefunctionalityhasbeenaddedtothewebinterface,makingiteasiertoworkwithmultipledocumentsatonce.Additionally,basicJSONimportandexporttoolshavebeenadded.

Command-lineoptionsadded

Thecommand-lineoption--javascript.v8-contextswasaddedtoarangodtoprovidebettercontroloverthenumberofV8contextscreatedinarangod.

Previously,thenumberofV8contextsarangodcreatedatstartupwasequaltothenumberofserverthreads(asspecifiedbyoption--server.threads).

InsomesituationsitmaybemoresensibletocreatedifferentamountsofthreadsandV8contexts.ThisisbecauseeachV8contextscreatedwillconsumememoryandrequiresCPUresourcesforperiodicgarbagecollection.Contrary,serverthreadsdonothavesuchhighmemoryorCPUfootprint.

Iftheoption--javascript.v8-contextsisnotspecified,thenumberofV8contextscreatedatstartupwillremainequaltothenumberofserverthreads.ThusnochangeinconfigurationisrequiredtokeepthesamebehaviorasinpreviousArangoDBversions.

Thecommand-lineoption--log.use-local-timewasaddedtoprintdatesandtimesinArangoDB'slogintheserver-localtimezoneinsteadofUTC.Ifitisnotset,thetimezonewilldefaulttoUTC.

Theoption--backslash-escapehasbeenaddedtoarangoimp.SpecifyingthisoptionwillusethebackslashastheescapecharacterforliteralquoteswhenparsingCSVfiles.Theescapecharacterforliteralquotesisstillthedoublequotecharacter.

ArangoDB'sbuilt-inHTTPservernowsupportsHTTPpipelining.

TheArangoShelltutorialfromthearangodb.comwebsiteisnowintegratedintotheArangoDBshell.

PowerfulFoxxEnhancementsWiththenewjobqueuefeatureyoucanrunasyncjobstocommunicatewithexternalservices,FoxxqueriesmakewritingcomplexAQLqueriesmucheasierandFoxxsessionswillhandletheauthenticationandsessionhassleforyou.

FoxxQueries

WritinglongAQLqueriesinJavaScriptcanquicklybecomeunwieldy.Asof2.3ArangoDBbundlestheArangoDBQueryBuildermodulethatprovidesaJavaScriptAPIforwritingcomplexAQLquerieswithoutstringconcatenation.Allbuilt-infunctionsthatacceptAQLstringsnowsupportquerybuilderinstancesdirectly.AdditionallyFoxxprovidesamethodFoxx.createQueryforcreatingparametrizedqueriesthatcanreturnFoxxmodelsorapplyarbitrarytransformationstothequeryresults.

FoxxSessions

ThesessionfunctionalityinFoxxhasbeencompletelyrewritten.TheoldactivateAuthenticationAPIisstillsupportedbutmaybedeprecatedinthefuture.ThenewactivateSessionsAPIsupportscookiesorconfigurableheaders,providesoptionalJSONWebTokenandcryptographicsigningsupportandusesthenewsessionsFoxxapp.

ArangoDB2.3providesFoxxappsforusermanagementandsaltedhash-basedauthenticationwhichcanbereplacedwithorsupplementedbyalternativeimplementations.Foranexampleappusingboththebuilt-inauthenticationandOAuth2seetheFoxxSessionsExampleapp.

FoxxQueues

FoxxnowprovidesasyncworkersviatheFoxxQueuesAPI.Jobsenqueuedinajobqueuewillbeexecutedasynchronouslyoutsideoftherequest/responsecycleofFoxxcontrollersandcanbeusedtocommunicatewithexternalservicesorperformtasksthattakealongtimetocompleteormayrequiremultipleattempts.

WhatsNewin2.3

Jobscanbescheduledinadvanceorsettobeexecutedimmediately,thenumberofretryattempts,theretrydelayaswellassuccessandfailurehandlerscanbedefinedforeachjobindividually.Jobtypesthatintegratevariousexternalservicesfortransactionale-mails,loggingandusertrackingcanbefoundintheFoxxappregistry.

TherequestandresponseobjectsinFoxxcontrollersnowprovidemethodsforreadingandwritingrawcookiesandsignedcookies.

MountedFoxxappswillnowbeloadedwhenarangodstartsratherthanatthefirstdatabaserequest.Thismayresultinslightlyslowerstartuptimes(butafasterresponseforthefirstrequest).

WhatsNewin2.3

Configurationfilechanges

Threadsandcontexts

Thenumberofserverthreadsspecifiedisnowtheminimumofthreadsstarted.Therearesituationinwhichthreadsarewaitingforresultsofdistributeddatabaseservers.Inthiscasethenumberofthreadsisdynamicallyincreased.

WithArangoDB2.3,thenumberofserverthreadscanbeconfiguredindependentlyofthenumberofV8contexts.Theconfigurationoption--javascript.v8-contextswasaddedtoarangodtoprovidebettercontroloverthenumberofV8contextscreatedinarangod.

Previously,thenumberofV8contextsarangodcreatedatstartupwasequaltothenumberofserverthreads(asspecifiedbyoption--server.threads).

InsomesituationsitmaybemoresensibletocreatedifferentamountsofthreadsandV8contexts.ThisisbecauseeachV8contextscreatedwillconsumememoryandrequiresCPUresourcesforperiodicgarbagecollection.Contrary,serverthreadsdonothavesuchhighmemoryorCPUfootprint.

Iftheoption--javascript.v8-contextsisnotspecified,thenumberofV8contextscreatedatstartupwillremainequaltothenumberofserverthreads.ThusnochangeinconfigurationisrequiredtokeepthesamebehaviorasinpreviousArangoDBversions.

Ifyouareusingthedefaultconfigfilesormergethemwithyourlocalconfigfiles,pleasereviewifthedefaultnumberofserverthreadsisokayinyourenvironment.AdditionallyyoushouldverifythatthenumberofV8contextscreated(asspecifiedinoption--javascript.v8-contexts)isokay.

Syslog

Thecommand-lineoption--log.syslogwasusedinpreviousversionsofArangoDBtoturnloggingtosyslogonoroff:whensettingtoanon-emptystring,syslogloggingwasturnedon,otherwiseturnedoff.Whensyslogloggingwasturnedon,loggingwasdonewiththeapplicationnamespecifiedin--log.application,whichdefaultedtotriagens.Therewasalsoacommand-lineoption--log.hostnamewhichcouldbesetbutdidnothaveanyeffect.

Thisbehaviorturnedouttobeunintuitiveandwaschangedin2.3asfollows:

thecommand-lineoption--log.syslogisdeprecatedanddoesnothaveanyeffectwhenstartingArangoDB.toturnonsyslogloggingin2.3,theoption--log.facilityhastobesettoanon-emptystring.ThevalueforfacilityisOS-dependent(possiblevaluescanbefoundin/usr/include/syslog.horthelike-usershouldbeavailableonmanysystems).thedefaultvaluefor--log.applicationhasbeenchangedfromtriagenstoarangod.thecommand-lineoption--log.hostnameisdeprecatedanddoesnothaveanyeffectwhenstartingArangoDB.Instead,thehostnamewillbesetbysyslogautomatically.whenloggingtosyslog,ArangoDBnowomitsthedatetimeprefixandtheprocessid,becausethey'llbeaddedbysyslogautomatically.

AQLqueriesthrowlessexceptions

ArangoDB2.3containsacompletelyrewrittenAQLqueryoptimizerandexecutionengine.ThismeansthatAQLquerieswillbeexecutedwithadifferentenginethaninArangoDB2.2andearlier.PartsofAQLqueriesmightbeexecutedindifferentorderthanbeforebecausetheAQLoptimizerhasmorefreedomtomovethingsaroundinaquery.

InpreviousversionsofArangoDB,AQLqueriesabortedwithanexceptioninmanysituationsandthrewaruntimeexception.ExceptionswerethrownwhentryingtofindavalueusingtheINoperatorinanon-arrayelement,whentryingtousenon-booleanvalueswiththelogicaloperands&&or||or!,whenusingnon-numericvaluesinarithmeticoperations,whenpassingwrongparametersintofunctionsetc.

InArangoDB2.3thishasbeenchangedinmanycasestomakeAQLmoreuser-friendlyandtoallowtheoptimizationtoperformmuchmorequeryoptimizations.

Hereisasummaryofchanges:

whenanon-arrayvalueisusedontheright-handsideoftheINoperator,theresultwillbefalseinArangoDB2.3,andnoexceptionwillbethrown.thebooleanoperators&&and||donotthrowinArangoDB2.3ifanyoftheoperandsisnotabooleanvalue.Instead,theywillperformanimplicitcastofthevaluestobooleans.Theirresultwillbeasfollows:

lhs&&rhswillreturnlhsifitisfalseorwouldbefalsewhenconvertedintoaboolean.Iflhsistrueorwouldbetruewhenconvertedtoaboolean,rhswillbereturned.lhs||rhswillreturnlhsifitistrueorwouldbetruewhenconvertedintoaboolean.Iflhsisfalseorwouldbefalsewhenconvertedtoaboolean,rhswillbereturned.!valuewillreturnthenegatedvalueofvalueconvertedintoaboolean

thearithmeticoperators(+,-,*,/,%)canbeappliedtoanyvalueandwillnotthrowexceptionswhenappliedtonon-numericvalues.Instead,anyvalueusedintheseoperatorswillbecastedtoanumericvalueimplicitly.Ifnonumericresultcanbeproducedbyanarithmeticoperator,itwillreturnnullinArangoDB2.3.Thisisalsotruefordivisionbyzero.passingargumentsofinvalidtypesintoAQLfunctionsdoesnotthrowaruntimeexceptioninmostcases,butmayproduceruntimewarnings.Built-inAQLfunctionsthatreceiveinvalidargumentswillthenreturnnull.

NestedFORloopexecutionorder

Thequeryoptimizerin2.3maypermutetheorderofnestedFORloopsinAQLqueries,providedthatexchangingtheloopswillnotalteraqueryresult.However,achangeintheorderofreturnedvaluesisallowedbecausenosortorderisguaranteedbyAQL(andwasnever)unlessanexplicitSORTstatementisusedinaquery.

ChangedreturnvaluesofArangoQueryCursor.getExtra()

ThereturnvalueofArangoQueryCursor.getExtra()hasbeenchangedinArangoDB2.3.Itnowcontainsastatsattributewithstatisticsaboutthequerypreviouslyexecuted.Italsocontainsawarningsattributewithwarningsthathappenedduringqueryexecution.Thereturnvaluestructurehasbeenunifiedin2.3forbothread-onlyanddata-modificationqueries.

Thereturnvaluelookslikethisforaread-onlyquery:

arangosh>stmt=db._createStatement("FORiINmycollectionRETURNi");stmt.execute().getExtra()

"stats":{

"writesExecuted":0,

"writesIgnored":0,

"scannedFull":2600,

"scannedIndex":0

"warnings":[]

Fordata-modificationqueries,ArangoDB2.3returnsaresultwiththesamestructure:

arangosh>stmt=db._createStatement("FORiINxxREMOVEiINxx");stmt.execute().getExtra()

"stats":{

"writesExecuted":2600,

"writesIgnored":0,

"scannedFull":2600,

"scannedIndex":0

"warnings":[]

InArangoDB2.2,thereturnvalueofArangoQueryCursor.getExtra()wasemptyforread-onlyqueriesandcontainedanattributeoperationswithtwosub-attributesfordata-modificationqueries:

arangosh>stmt=db._createStatement("FORiINmycollectionRETURNi");stmt.execute().getExtra()

arangosh>stmt=db._createStatement("FORiINmycollectionREMOVEiINmycollection");stmt.execute().getExtra()

"operations":{

"executed":2600,

"ignored":0

ChangedreturnvaluesinHTTPmethodPOST/_api/cursor

ThepreviouslymentionedchangealsoleadstothestatisticsbeingreturnedintheHTTPRESTAPImethodPOST/_api/cursor.Previously,thereturnvaluecontainedanoptionalextraattributethatwasfilledonlyfordata-modificationqueriesandinsomeothercasesasfollows:

"result":[],

"hasMore":false,

"extra":{

"operations":{

"executed":2600,

"ignored":0

WiththechangedresultstructureinArangoDB2.3,theextraattributeintheresultwilllooklikethis:

"result":[],

"hasMore":false,

"extra":{

"stats":{

"writesExecuted":2600,

"writesIgnored":0,

"scannedFull":0,

"scannedIndex":0

"warnings":[]

IfthequeryoptionfullCountisrequested,thefullCountresultvaluewillalsobereturnedinsidethestatsattributeoftheextraattribute,andnotdirectlyasanattributeinsidetheextraattributeasin2.2.NotethatafullCountwillonlybepresentinextra.statsifitwasrequestedasanoptionforthequery.

TheresultinArangoDB2.3willalsocontainawarningsattributewiththearrayofwarningsthathappenedduringqueryexecution.

ChangedreturnvaluesinArangoStatement.explain()

ThereturnvalueofArangoStatement.explain()haschangedsignificantlyinArangoDB2.3.Thenewreturnvaluestructureisnotcompatiblewiththestructurereturnedby2.2.

InArangoDB2.3,thefullexecutionplanforanAQLqueryisreturnedalongsideallappliedoptimizerrules,optimizationwarningsetc.Itisalsopossibletohavetheoptimizerreturnallexecutionplans.Thisrequiredanewdatastructure.

ClientprogramsthatuseArangoStatement.explain()ortheHTTPRESTAPImethodPOST/_api/explainmayneedtobeadjustedtousethenewreturnformat.

ThereturnvalueofArangoStatement.parse()hasbeenextendedinArangoDB2.3.Inadditiontotheexistingattributes,ArangoDB2.3willalsoreturnanastattributecontainingtheabstractsyntaxtreeofthestatement.Thisextraattributecansafelybeignoredbyclientprograms.

Variablesnotupdatableinqueries

PreviousversionsofArangoDBallowedthemodificationofvariablesinsideAQLqueries,e.g.

LETcounter=0

FORiIN1..10

LETcounter=counter+1

RETURNcounter

Whilethisisadmittedlyaconvenientfeature,thenewqueryoptimizerdesigndidnotallowtokeepit.Additionally,updatingvariablesinsideaquerywouldpreventalotofoptimizationstoqueriesthatwewouldliketheoptimizertomake.Additionally,updatingvariablesinqueriesthatrunondifferentnodesinaclusterwouldlikecausenon-deterministicbehaviorbecausequeriesarenotexecutedlinearly.

ChangedreturnvalueofTO_BOOL

TheAQLfunctionTO_BOOLnowalwaysreturnstrueifitsargumentisanarrayoranobject.InpreviousversionsofArangoDB,thefunctionreturnedfalseforemptyarraysorforobjectswithoutattributes.

ChangedreturnvalueofTO_NUMBER

TheAQLfunctionTO_NUMBERnowreturnsnullifitsargumentisanobjectoranarraywithmorethanonemember.InpreviousversionofArangoDB,thereturnvalueinthesecaseswas0.TO_NUMBERwillreturn0foremptyarray,andthenumericequivalentofthearraymember'svalueforarrayswithasinglemember.

NewAQLkeywords

ThefollowingkeywordshavebeenaddedtoAQLinArangoDB2.3:

NOTANDOR

UnquotedusageofthesekeywordsforattributenamesinAQLquerieswilllikelyfailinArangoDB2.3.Ifanysuchattributenameneedstobeusedinaquery,itshouldbeenclosedinbacktickstoindicatetheusageofaliteralattributename.

Removedfeatures

Bitarrayindexes

Bitarrayindexeswereonlyhalf-waydocumentedandintegratedinpreviousversionsofArangoDBsotheirbenefitwaslimited.ThesupportforbitarrayindexeshasthusbeenremovedinArangoDB2.3.Itisnotpossibletocreateindexesoftype"bitarray"withArangoDB2.3.

WhenacollectionisopenedthatcontainsabitarrayindexdefinitioncreatedwithapreviousversionofArangoDB,ArangoDBwillignoreitandlogthefollowingwarning:

indextype'bitarray'isnotsupportedinthisversionofArangoDBandisignored

FutureversionsofArangoDBmayautomaticallyremovesuchindexdefinitionssothewarningswilleventuallydisappear.

Otherremovedfeatures

TheHTTPRESTAPImethodatPOST/_admin/modules/flushhasbeenremoved.

Knownissues

InArangoDB2.3.0,AQLqueriescontainingfilterconditionswithanINexpressionwillnotyetuseanindex:

FORdocINcollectionFILTERdoc.indexedAttributeIN[...]RETURNdoc

FORdocINcollection

FILTERdoc.indexedAttributeIN[...]

RETURNdoc

We’recurrentlyworkingongettingtheINoptimizationsdone,andwillshipthemina2.3maintenancereleasesoon(e.g.2.3.1or2.3.2).

AQLimprovements

DatamodificationAQLqueries

Uptoincludingversion2.1,AQLsupporteddataretrievaloperationsonly.StartingwithArangoDBversion2.2,AQLalsosupportsthefollowingdatamodificationoperations:

INSERT:insertnewdocumentsintoacollectionUPDATE:partiallyupdateexistingdocumentsinacollectionREPLACE:completelyreplaceexistingdocumentsinacollectionREMOVE:removeexistingdocumentsfromacollection

Data-modificationoperationsarenormallycombinedwithotherAQLstatementssuchasFORloopsandFILTERconditionstodeterminethesetofdocumentstooperateon.Forexample,thefollowingquerywillfindalldocumentsincollectionusersthatmatchaspecificconditionandsettheirstatusvariabletoinactive:

FORuINusers

FILTERu.status=='notactive'

UPDATEuWITH{status:'inactive'}INusers

Thefollowingquerycopiesalldocumentsfromcollectionusersintocollectionbackup:

FORuINusers

INSERTuINbackup

Andthisqueryremovesdocumentsfromcollectionbackup:

FORdocINbackup

FILTERdoc.lastModified<DATE_NOW()-3600

REMOVEdocINbackup

Formoreinformationondata-modificationqueries,pleaserefertoDatamodificationqueries.

Updatablevariables

Previously,thevalueofavariableassignedinanAQLquerywiththeLETkeywordwasnotupdatableinanAQLquery.Thispreventedstatementslikethefollowingfrombeingexecutable:

LETsum=0

FORvINvalues

SORTv.year

LETsum=sum+v.value

RETURN{year:v.year,value:v.value,sum:sum}

OtherAQLimprovements

addedAQLTRANSLATEfunction

Thisfunctioncanbeusedtoperformlookupsfromstaticobjects,e.g.

LETcountryNames={US:"UnitedStates",UK:"UnitedKingdom",FR:"France"}

RETURNTRANSLATE("FR",countryNames)

WhatsNewin2.2

LETlookup={foo:"foo-replacement",bar:"bar-replacement",baz:"baz-replacement"}

RETURNTRANSLATE("foobar",lookup,"notcontained!")

Write-aheadlogAllwriteoperationsinanArangoDBserverwillnowbeautomaticallyloggedintheserver'swrite-aheadlog.Thewrite-aheadlogisasetofappend-onlylogfiles,anditisusedincaseofacrashrecoveryandforreplication.

Datafromthewrite-aheadlogwilleventuallybemovedintothejournalsordatafilesofcollections,allowingtheservertoremoveolderwrite-aheadlogfiles.

Cross-collectiontransactionsinArangoDBshouldbenefitconsiderablybythischange,aslesswritesthaninpreviousversionsarerequiredtoensurethedataofmultiplecollectionsareatomicallyanddurablycommitted.Alldata-modifyingoperationsinsidetransactions(insert,update,remove)willwritetheiroperationsintothewrite-aheadlogdirectlynow.Inpreviousversions,suchoperationswerebuffereduntilthecommitorrollbackoccurred.TransactionswithmultipleoperationsshouldthereforerequirelessphysicalmemorythaninpreviousversionsofArangoDB.

Thedatainthewrite-aheadlogcanalsobeusedinthereplicationcontext.InpreviousversionsofArangoDB,replicatingfromamasterrequiredturningonaspecialreplicationloggeronthemaster.Thereplicationloggercausedanextrawriteoperationintothe_replicationsystemcollectionforeachactualwriteoperation.Thisextrawriteisnowsuperfluous.Instead,slavescanreaddirectlyfromthemaster'swrite-aheadlogtogetinformedaboutmostrecentdatachanges.Thisremovestheneedtostoredata-modificationoperationsinthe_replicationcollectionaltogether.

Fortheconfigurationofthewrite-aheadlog,pleaserefertoWrite-aheadlogoptions.

Theintroductionofthewrite-aheadlogalsoremovestheneedtoconfigureandstartthereplicationloggeronamaster.ThoughthereplicationloggerobjectisstillavailableinArangoDB2.2toensureAPIcompatibility,starting,stopping,orconfiguringitwillhavenoeffect.

PerformanceimprovementsRemovedsortingofattributenameswhenincollectionshaper

InpreviousversionsofArangoDB,addingadocumentwithpreviouslynot-usedattributenamescausedafullsortofallattributenamesusedinthecollection.Thesortingwasdonetoensurefastcomparisonsofattributenamesinsomerareedgecases,butitconsiderablysloweddowninsertsintocollectionswithmanydifferentorevenuniqueattributenames.

Specializedprimaryindeximplementationtoallowfasterhashtablerebuildingandreducelookupsindatafilesfortheactualvalueof_key.Thisalsoreducestheamountofrandommemoryaccessesforprimaryindexinserts.

Reclamationofindexmemorywhendeletinglastdocumentincollection

Deletingdocumentsfromacollectiondidnotleadtoindexsizesbeingreduced.Instead,theindexmemorywaskeptallocatedandre-usedlaterwhenacollectionwasrefilledwithnewdocuments.Now,indexmemoryofprimaryindexesandhashindexesisreclaimedinstantlywhenthelastdocumentinacollectionisremoved.

Preventbufferingoflongprintresultsinarangosh'sandarangod'sprintcommand

Thischangewillemitbufferedintermediateprintresultsanddiscardtheoutputbuffertoquicklydeliverprintresultstotheuser,andtopreventconstructingverylargebuffersforlargeresults.

Addedinsertmethodasanaliasforsave.Documentscannowbeinsertedintoacollectionusingeithermethod:

db.test.save({foo:"bar"});

db.test.insert({foo:"bar"});

Cleanupofoptionsfordata-modificationoperations

WhatsNewin2.2

Manyofthedata-modificationoperationshadsignatureswithmanyoptionalboolparameters,e.g.:

db.test.update("foo",{bar:"baz"},true,true,true)

db.test.replace("foo",{bar:"baz"},true,true)

db.test.remove("foo",true,true)

db.test.save({bar:"baz"},true)

Suchlongparameterlistswereunintuitiveandhardtousewhenonlyoneoftheoptionalparametersshouldhavebeenset.

TomaketheAPIsmoreusable,theoperationsnowunderstandthefollowingalternativesignature:

collection.update(key,update-document,options)

collection.replace(key,replacement-document,options)

collection.remove(key,options)

collection.save(document,options)

Examples:

db.test.update("foo",{bar:"baz"},{overwrite:true,keepNull:true,waitForSync:true})

db.test.replace("foo",{bar:"baz"},{overwrite:true,waitForSync:true})

db.test.remove("foo",{overwrite:true,waitForSync:true})

db.test.save({bar:"baz"},{waitForSync:true})

Added--overwriteoptiontoarangoimp

Thisallowsremovingalldocumentsinacollectionbeforeimportingintoitusingarangoimp.

Honorstartupoption--server.disable-statisticswhendecidingwhetherornottostartperiodicstatisticscollectionjobs

Previously,thestatisticscollectionjobswerestartedeveniftheserverwasstartedwiththe--server.disable-statisticsflagbeingsettotrue.Nowiftheoptionissettotrue,nostatisticswillbecollectedontheserver.

DisallowstoringofJavaScriptobjectsthatcontainJavaScriptnativeobjectsoftypeDate,Function,RegExporExternal,e.g.

db.test.save({foo:/bar/});

db.test.save({foo:newDate()});

Thiswillnowprint

Error:<data>cannotbeconvertedintoJSONshape:couldnotshapedocument

Previously,objectsofthesetypesweresilentlyconvertedintoanemptyobject(i.e.{})andnowarningwasissued.

Tostoresuchobjectsinacollection,explicitlyconvertthemintostringslikethis:

db.test.save({foo:String(/bar/)});

db.test.save({foo:String(newDate())});

Removedfeatures

MRubyintegrationforarangod

ArangoDBhadanexperimentalMRubyintegrationinsomeofthepublishbuilds.Thiswasn'tcontinuouslydeveloped,andsoithasbeenremovedinArangoDB2.2.

Thischangehasledtothefollowingstartupoptionsbeingsuperfluous:

--ruby.gc-interval

--ruby.action-directory

--ruby.modules-path

--ruby.startup-directory

WhatsNewin2.2

SpecifyingthesestartupoptionswilldonothinginArangoDB2.2,sousingtheseoptionsshouldbeavoidedfromnowonastheymightberemovedinafutureversionofArangoDB.

Removedstartupoptions

ThefollowingstartupoptionshavebeenremovedinArangoDB2.2.Specifyingthemintheserver'sconfigurationfilewillnotproduceanerrortomakemigrationeasier.Still,usageoftheseoptionsshouldbeavoidedastheywillnothaveanyeffectandmightfullyberemovedinafutureversionofArangoDB:

--database.remove-on-drop

--database.force-sync-properties

--random.no-seed

--ruby.gc-interval

--ruby.action-directory

--ruby.modules-path

--ruby.startup-directory

--server.disable-replication-logger

MultiCollectionGraphs

ArangoDBisamultimodeldatabasewithnativegraphsupport.Inversion2.2thefeaturesforgraphshavebeenimprovedbyintegrationofanewgraphmodule.Allgraphscreatedwiththeoldmoduleareautomaticallymigratedintothenewmodulebutcanstillbeusedbytheoldmodule.

Newgraphmodule

Uptoincludingversion2.1,ArangoDBofferedamoduleforgraphsandgraphoperations.Thismoduleallowedyoutouseexactlyoneedgecollectiontogetherwithonevertexcollectioninagraph.WithArangoDBversion2.2thisgraphmoduleisdeprecatedandanewgraphmoduleisoffered.Thisnewmoduleallowstocombineanarbitrarynumberofvertexcollectionsandedgecollectionsinthesamegraph.Foreachedgecollectionalistofcollectionscontainingsourceverticesandalistofcollectionscontainingtargetverticescanbedefined.IfanedgeisstoredArangoDBchecksifthisedgeisvalidinthiscollection.Furthermoreifavertexisremovedfromoneofthecollectionsallconnectededgeswillberemovedaswell,givingtheguaranteeofnolooseendsinthegraphs.Thelayoutofthegraphcanbemodifiedatruntimebyaddingorremovingcollectionsandchangingthedefinitionsforedgecollections.Alloperationsonthegraphlevelaretransactionalbydefault.

GraphsinAQL

MulticollectiongraphshavebeenaddedtoAQLaswell.Basicfunctionality(gettingvertices,edges,neighbors)canbeexecutedusingtheentiregraph.Alsomoreadvancedfeatureslikeshortestpathcalculations,characteristicfactorsofthegraphortraversalshavebeenintegratedintoAQL.Forthesefunctionsallgraphscreatedwiththegraphmodulecanbeused.

WhatsNewin2.2

NewEdgesIndex

Theedgesindex(usedtostoreconnectionsbetweennodesinagraph)internallyusesanewdatastructure.Thisdatastructureimprovestheperformancewhenpopulatingtheedgeindex(i.e.whenloadinganedgecollection).Forlargegraphsloadingcanbe20timesfasterthanwithArangoDB2.0.

Additionally,thenewindexfixesperformanceproblemsthatoccurredwhenmanyduplicate_fromor_tovalueswerecontainedintheindex.Furthermore,thenewindexsupportsfasterremovalofedges.

Finally,whenloadinganexistingcollectionandbuildingtheedgesindexforthecollection,lessmemoryre-allocationswillbeperformed.

Overall,thisshouldconsiderablyspeeduploadingedgecollections.

Thenewindextypereplacestheoldedgesindextypeautomatically,withoutanychangesbeingrequiredbytheenduser.

TheAPIofthenewindexiscompatiblewiththeAPIoftheoldindex.Stillitispossiblethatthenewindexreturnsedgesinadifferentorderthantheoldindex.Thisisstillconsideredtobecompatiblebecausetheoldindexhadneverguaranteedanyresultordereither.

AQLImprovements

AQLoffersfunctionalitytoworkwithdates.DatesarenodatatypesoftheirowninAQL(neithertheyareinJSON,whichisoftenusedasaformattoshipdataintoandoutofArangoDB).Instead,datesinAQLareinternallyrepresentedbyeithernumbers(timestamps)orstrings.ThedatefunctionsinAQLprovidemechanismstoconvertfromanumerictimestamptoastringrepresentationandviceversa.

TherearetwodatefunctionsinAQLtocreatedatesforfurtheruse:

DATE_TIMESTAMP(date)CreatesaUTCtimestampvaluefromdate

DATE_TIMESTAMP(year,month,day,hour,minute,second,millisecond):Sameasbefore,butallowsspecifyingtheindividualdatecomponentsseparately.Allparametersafterdayareoptional.

DATE_ISO8601(date):ReturnsanISO8601datetimestringfromdate.ThedatetimestringwillalwaysuseUTCtime,indicatedbytheZatitsend.

DATE_ISO8601(year,month,day,hour,minute,second,millisecond):sameasbefore,butallowsspecifyingtheindividualdatecomponentsseparately.Allparametersafterdayareoptional.

Thesetwoabovedatefunctionsacceptthefollowinginputvalues:

numerictimestamps,indicatingthenumberofmillisecondselapsedsincetheUNIXepoch(i.e.January1st197000:00:00UTC).Anexampletimestampvalueis1399472349522,whichtranslatesto2014-05-07T14:19:09.522Z.

datetimestringsinformatsYYYY-MM-DDTHH:MM:SS.MMM,YYYY-MM-DDHH:MM:SS.MMM,orYYYY-MM-DD.Millisecondsarealwaysoptional.

Atimezonedifferencemayoptionallybeaddedattheendofthestring,withthehoursandminutesthatneedtobeaddedorsubtractedtothedatetimevalue.Forexample,2014-05-07T14:19:09+01:00canbeusedtospecifyaonehouroffset,and2014-05-07T14:19:09+07:30canbespecifiedforsevenandhalfhoursoffset.Negativeoffsetsarealsopossible.Alternativelytoanoffset,aZcanbeusedtoindicateUTC/Zulutime.

Anexamplevalueis2014-05-07T14:19:09.522ZmeaningMay7th2014,14:19:09and522milliseconds,UTC/Zulutime.Anotherexamplevaluewithouttimecomponentis2014-05-07Z.

Pleasenotethatifnotimezoneoffsetisspecifiedinadatestring,ArangoDBwillassumeUTCtimeautomatically.Thisisdonetoensureportabilityofqueriesacrossserverswithdifferenttimezonesettings,andbecausetimestampswillalwaysbeUTC-based.

WhatsNewin2.1

individualdatecomponentsasseparatefunctionarguments,inthefollowingorder:

yearmonthdayhourminutesecondmillisecond

Allcomponentsfollowingdayareoptionalandcanbeomitted.Notethatnotimezoneoffsetscanbespecifiedwhenusingseparatedatecomponents,andUTC/Zulutimewillbeused.

ThefollowingcallstoDATE_TIMESTAMPareequivalentandwillallreturn1399472349522:

DATE_TIMESTAMP("2014-05-07T14:19:09.522")

DATE_TIMESTAMP("2014-05-07T14:19:09.522Z")

DATE_TIMESTAMP("2014-05-0714:19:09.522")

DATE_TIMESTAMP("2014-05-0714:19:09.522Z")

DATE_TIMESTAMP(2014,5,7,14,19,9,522)

DATE_TIMESTAMP(1399472349522)

ThesameistrueforcallstoDATE_ISO8601thatalsoacceptsvariableinputformats:

DATE_ISO8601("2014-05-07T14:19:09.522Z")

DATE_ISO8601("2014-05-0714:19:09.522Z")

DATE_ISO8601(2014,5,7,14,19,9,522)

DATE_ISO8601(1399472349522)

Theabovefunctionsareallequivalentandwillreturn"2014-05-07T14:19:09.522Z".

ThefollowingdatefunctionscanbeusedwithdatescreatedbyDATE_TIMESTAMPandDATE_ISO8601:

DATE_DAYOFWEEK(date):Returnstheweekdaynumberofdate.Thereturnvalueshavethefollowingmeanings:

0:Sunday1:Monday2:Tuesday3:Wednesday4:Thursday5:Friday6:Saturday

DATE_YEAR(date):Returnstheyearpartofdateasanumber.

DATE_MONTH(date):Returnsthemonthpartofdateasanumber.

DATE_DAY(date):Returnsthedaypartofdateasanumber.

DATE_HOUR(date):Returnsthehourpartofdateasanumber.

DATE_MINUTE(date):Returnstheminutepartofdateasanumber.

DATE_SECOND(date):Returnsthesecondspartofdateasanumber.

DATE_MILLISECOND(date):Returnsthemillisecondspartofdateasanumber.

Thefollowingotherdatefunctionsarealsoavailable:

DATE_NOW():Returnsthecurrenttimeasatimestamp.

Notethatthisfunctionisevaluatedoneveryinvocationandmayreturndifferentvalueswheninvokedmultipletimesinthesamequery.

ThefollowingotherAQLfunctionshavebeenaddedinArangoDB2.1:

FLATTEN:thisfunctioncanturnanarrayofsub-arraysintoasingleflatarray.Allarrayelementsintheoriginalarraywillbeexpandedrecursivelyuptoaconfigurabledepth.Theexpandedvalueswillbeaddedtothesingleresultarray.

WhatsNewin2.1

Example:

FLATTEN([1,2,[3,4],5,[6,7],[8,[9,10]])

willexpandthesub-arraysonthefirstlevelandproduce:

[1,2,3,4,5,6,7,8,[9,10]]

Tofullyflattenthearray,themaximumdepthcanbespecified(e.g.withavalueof2):

FLATTEN([1,2,[3,4],5,[6,7],[8,[9,10]],2)

Thiswillfullyexpandthesub-arraysandproduce:

[1,2,3,4,5,6,7,8,9,10]

CURRENT_DATABASE:thisfunctionwillreturnthenameofthedatabasethecurrentqueryisexecutedin.

CURRENT_USER:thisfunctionreturnsthenameofthecurrentuserthatisexecutingthequery.Ifauthorizationisturnedofforthequeryisexecutedoutsideofarequestcontext,nouserispresentandthefunctionwillreturnnull.

ClusterDumpandRestore

Thedumpandrestoretools,arangodumpandarangorestore,cannowbeusedtodumpandrestorecollectionsinacluster.Additionally,acollectiondumpfromastandaloneArangoDBservercanbeimportedintoacluster,andviceversa.

WebInterfaceImprovements

Thewebinterfaceinversion2.1hasamorecompactdashboard.Itprovideschartswithtime-seriesforincomingrequests,HTTPtransfervolumeandsomeserverresourceusagefigures.

Additionallyitprovidestrendindicators(e.g.15minaverages)anddistributioncharts(akahistogram)forsomefigures.

FoxxImprovements

ToeasilyaccessafileinsidethedirectoryofaFoxxapplicationfromwithinFoxx,Foxx'sapplicationContextnowprovidesthefoxxFilename()function.Itcanbeusedtoassemblethefullfilenameofafileinsidetheapplication'sdirectory.TheapplicationContextcanbeaccessedasglobalvariablefromanymodulewithinaFoxxapplication.

ThefilenamecanbeusedinsideFoxxactionsorsetup/teardownscripts,e.g.topopulateaFoxxapplication'scollectionwithdata.

TherequirefunctionnowalsopreferslocalmoduleswhenusedfrominsideaFoxxapplication.ThisallowsputtingmodulesinsidetheFoxxapplicationdirectoryandrequiringthemeasily.Italsoallowsusingapplication-specificversionsoflibrariesthatarebundledwithArangoDB(suchasunderscore.js).

WindowsInstaller

TheWindowsinstallershippedwithArangoDBnowsupportsinstallationofArangoDBforthecurrentuserorallusers,withtherequiredprivileges.ItalsosupportstheinstallationofArangoDBasaservice.

Fixesfor32bitsystems

SeveralissueshavebeenfixedthatoccurredonlywhenusingArangoDBona32bitsoperatingsystem,specifically:

acrashinathirdpartycomponentusedtomanageclusterdata

WhatsNewin2.1

athirdpartylibrarythatfailedtoinitializeon32bitWindows,makingarangodandarangoshcrashimmediately.

overflowsofvaluesusedfornanosecond-precisiontimeouts:theseoverflowshaveledtoinvalidvaluesbeingpassedtosocketoperations,makingthemfailandre-trytoooften

Updateddrivers

SeveraldriversforArangoDBhavebeencheckedforcompatibilitywith2.1.Thecurrentlistofdriverswithcompatibilitynotescanbefoundonlinehere.

C++11usage

WehavemovedseveralfilesfromCtoC++,allowingmorecodereuseandreducingtheneedforshippingdatabetweenthetwo.WehavealsodecidedtorequireC++11supportforArangoDB,whichallowsustousesomeofthesimplifications,featuresandguaranteesthatthisstandardhasinstock.

ThatalsomeansacompilerwithC++11supportisrequiredtobuildArangoDBfromsource.ForinstanceGNUCCofatleastversion4.8.

MiscellaneousImprovementsCancelableasynchronousjobs:severalpotentiallylong-runningjobscannowbecanceledviaanexplicitcanceloperation.Thisallowsstoppinglong-runningqueries,traversalsorscriptswithoutshuttingdownthecompleteArangoDBprocess.Jobcancelationisprovidedforasynchronouslyexecutedjobsasisdescribedin@refHttpJobCancel.

Server-sideperiodictaskmanagement:anArangoDBservernowprovidesfunctionalitytoregisterandunregisterperiodictasks.Tasksareuser-definedJavaScriptactionsthatcanberunperiodicallyandautomatically,independentofanyHTTPrequests.

Thefollowingtaskmanagementfunctionsareprovided:

require("org/arangodb/tasks").register():registersaperiodictaskrequire("org/arangodb/tasks").unregister():unregistersandremovesaperiodictaskrequire("org/arangodb/tasks").get():retrievesaspecifictasksorallexistingtasks

Anexampletask(tobeexecutedevery15seconds)canberegisteredlikethis:

vartasks=require("org/arangodb/tasks");

tasks.register({

name:"thisisanexampletaskwithparameters",

period:15,

command:function(params){

vargreeting=params.greeting;

vardata=JSON.stringify(params.data);

require('console').log('%sfromparametertask:%s',greeting,data);

params:{greeting:"hi",data:"howareyou?"}

Pleaserefertothesection@refTasksformoredetails.

Thefiguresmethodofacollectionnowreturnsdataaboutthecollection'sindexmemoryconsumption.Thereturnedvalueindexes.sizewillcontainthetotalamountofmemoryacquiredbyallindexesofthecollection.Thisfigurecanbeusedtoassessthememoryimpactofindexes.

CapitalizedHTTPresponseheaders:fromversion2.1,ArangoDBwillreturncapitalizedHTTPheadersbydefault,e.g.Content-Lengthinsteadofcontent-length.ThoughtheHTTPspecificationstatesthatheadersfieldnamearecase-insensitive,severalolderclienttoolsrelyonaspecificcaseinHTTPresponseheaders.ThischangesmakeArangoDBabitmorecompatiblewiththose.

Simplifiedusageofdb._createStatement():toeasilyrunanAQLquery,themethoddb._createStatementnowallowspassingtheAQLqueryasastring.Previouslyitrequiredtheusertopassanobjectwithaqueryattribute(whichthencontainedthequerystring).

WhatsNewin2.1

ArangoDBnowsupportsbothversions:

db._createStatement(queryString);

db._createStatement({query:queryString});

WhatsNewin2.1

AppendixReferences:Briefoverviewsoverinterfacesandobjects

db:thedbobjectcollection:thecollectionobjectcursor:thecursorobject

JavaScriptModules:Listofbuilt-inandsupportedJSmodulesDeprecated:FeaturesthatareconsideredobsoleteandmaygetremovedeventuallyErrorcodesandmeanings:ListofallpossibleerrorsthatcanbeencounteredGlossary:Disambiguationpage

Appendix

ReferencesThissectioncontainsthedocumentationforsomeoftheAPI'swhicharecommononthearangoshell,arangodserverandforFoxxApps.

References

The"db"ObjectThedbobjectisavailableinarangoshbydefault,andcanalsobeimportedandusedinFoxxservices.

db.namereturnsacollectionobjectforthecollectionname.

Thefollowingmethodsexistsonthe_dbobject:

Database

db._createDatabase(name,options,users)db._databases()db._dropDatabase(name,options,users)db._useDatabase(name)

Indexes

db._index(index)db._dropIndex(index)

Properties

db._id()db._isSystem()db._name()db._path()db._version()

Collection

db._collection(name)db._create(name)db._drop(name)db._truncate(name)

db._createStatement(query)db._query(query)db._explain(query)

Document

db._document(object)db._exists(object)db._remove(selector)db._replace(selector,data)db._update(selector,data)

The"collection"ObjectThefollowingmethodsexistonthecollectionobject(returnedbydb.name):

Collection

collection.checksum()collection.count()collection.drop()collection.figures()collection.load()collection.properties()collection.revision()collection.rotate()collection.toArray()collection.truncate()collection.type()collection.unload()

Indexes

collection.dropIndex(index)collection.ensureIndex(description)collection.getIndexes(name)collection.index(index)

Document

collection.all()collection.any()collection.closedRange(attribute,left,right)collection.document(object)collection.documents(keys)collection.edges(vertex-id)collection.exists(object)collection.firstExample(example)collection.inEdges(vertex-id)collection.insert(data)collection.edges(vertices)collection.iterate(iterator,options)collection.outEdges(vertex-id)collection.queryByExample(example)collection.range(attribute,left,right)collection.remove(selector)collection.removeByKeys(keys)collection.rename()collection.replace(selector,data)collection.replaceByExample(example,data)collection.save(data)collection.update(selector,data)collection.updateByExample(example,data)

collection

SequentialAccessandCursorsIfaqueryreturnsacursor(forexamplebycallingdb._query(...)),thenyoucanusehasNextandnexttoiterateovertheresultsetortoArraytoconvertittoanarray.

Ifthenumberofqueryresultsisexpectedtobebig,itispossibletolimittheamountofdocumentstransferredbetweentheserverandtheclienttoaspecificvalue.ThisvalueiscalledbatchSize.ThebatchSizecanoptionallybesetbeforeorwhenasimplequeryisexecuted.Iftheserverhasmoredocumentsthanshouldbereturnedinasinglebatch,theserverwillsetthehasMoreattributeintheresult.Itwillalsoreturntheidoftheserver-sidecursorintheidattributeintheresult.ThisidcanbeusedwiththecursorAPItofetchanyoutstandingresultsfromtheserveranddisposetheserver-sidecursorafterwards.

TheinitialbatchSizevaluecanbesetusingthesetBatchSizemethodthatisavailableforeachtypeofsimplequery,orwhenthesimplequeryisexecutedusingitsexecutemethod.IfnobatchSizevalueisspecified,theserverwillpickareasonabledefaultvalue.

HasNext

checksifthecursorisexhaustedcursor.hasNext()

ThehasNextoperatorreturnstrue,thenthecursorstillhasdocuments.Inthiscasethenextdocumentcanbeaccessedusingthenextoperator,whichwilladvancethecursor.

Examples

arangosh>vara=db._query("FORxINfiveRETURNx");

arangosh>while(a.hasNext())print(a.next());

returnsthenextresultdocumentcursor.next()

IfthehasNextoperatorreturnstrue,thentheunderlyingcursorofthesimplequerystillhasdocuments.Inthiscasethenextdocumentcanbeaccessedusingthenextoperator,whichwilladvancetheunderlyingcursor.Ifyouusenextonanexhaustedcursor,thenundefinedisreturned.

Examples

arangosh>db._query("FORxINfiveRETURNx").next();

SetBatchsize

setsthebatchsizeforanyfollowingrequestscursor.setBatchSize(number)

Setsthebatchsizeforqueries.Thebatchsizedetermineshowmanyresultsareatmosttransferredfromtheservertotheclientinonechunk.

GetBatchsize

returnsthebatchsizecursor.getBatchSize()

Returnsthebatchsizeforqueries.Ifthereturnedvalueisundefined,theserverwilldetermineasensiblebatchsizeforanyfollowingrequests.

ExecuteQuery

cursor

executesaqueryquery.execute(batchSize)

Executesasimplequery.IftheoptionalbatchSizevalueisspecified,theserverwillreturnatmostbatchSizevaluesinoneroundtrip.ThebatchSizecannotbeadjustedafterthequeryisfirstexecuted.

Note:Thereisnoneedtoexplicitlycalltheexecutemethodifanothermeansoffetchingthequeryresultsischosen.Thefollowingtwoapproachesleadtothesameresult:

arangosh>result=db.users.all().toArray();

arangosh>varq=db._query("FORxINusersRETURNx");

........>result=[];

........>while(q.hasNext()){

........>result.push(q.next());

........>}

showexecutionresultsThefollowingtwoalternativesbothuseabatchSizeandreturnthesameresult:

arangosh>q=db.users.all();q.setBatchSize(20);q.execute();while(q.hasNext()){

print(q.next());}

arangosh>q=db.users.all();q.execute(20);while(q.hasNext()){print(q.next());}

Dispose

disposestheresultcursor.dispose()

Ifyouarenolongerinterestedinanyfurtherresults,youshouldcalldisposeinordertofreeanyresourcesassociatedwiththecursor.Aftercallingdisposeyoucannolongeraccessthecursor.

countsthenumberofdocumentscursor.count()

Thecountoperatorcountsthenumberofdocumentintheresultsetandreturnsthatnumber.Thecountoperatorignoresanylimitsandreturnsthetotalnumberofdocumentsfound.

Note:Notallsimplequeriessupportcounting.Inthiscasenullisreturned(Simplequeriesaredeprecated).

cursor.count(true)

Iftheresultsetwaslimitedbythelimitoperatorordocumentswereskipedusingtheskipoperator,thecountoperatorwithargumenttruewillusethenumberofelementsinthefinalresultset-afterapplyinglimitandskip.

Note:Notallsimplequeriessupportcounting.Inthiscasenullisreturned(Simplequeriesaredeprecated)..

cursor

JavaScriptModulesArangoDBusesaNode.jscompatiblemodulesystem.Youcanusethefunctionrequireinordertoloadamoduleorlibrary.Itreturnstheexportedvariablesandfunctionsofthemodule.

Theglobalvariablesglobal,process,console,Buffer,__filenameand__dirnameareavailablethroughoutArangoDBandFoxx.

Nodecompatibilitymodules

ArangoDBsupportsanumberofmodulesforcompatibilitywithNode.js,including:

assertimplementsbasicassertionandtestingfunctions.

bufferimplementsabinarydatatypeforJavaScript.

consoleisawellknownloggingfacilitytoalltheJavaScriptdevelopers.ArangoDBimplementsmostoftheConsoleAPI,withtheexceptionsofprofileandcount.

eventsimplementsaneventemitter.

fsprovidesafilesystemAPIforthemanipulationofpaths,directories,files,links,andtheconstructionoffilestreams.ArangoDBimplementsmostFilesystem/Afunctions.

moduleprovidesdirectaccesstothemodulesystem.

pathimplementsfunctionsdealingwithfilenamesandpaths.

punycodeimplementsconversionfunctionsforpunycodeencoding.

querystringprovidesutilitiesfordealingwithquerystrings.

streamprovidesastreaminginterface.

string_decoderimplementslogicfordecodingbuffersintostrings.

urlprovidesutilitiesforURLresolutionandparsing.

utilprovidesgeneralutilityfunctionslikeformatandinspect.

AdditionallyArangoDBprovidespartialimplementationsforthefollowingmodules:

net:onlyisIP,isIPv4andisIPv6.

process:onlyenvandcwd;stubsforargv,stdout.isTTY,stdout.write,nextTick.

timers:stubsforsetImmediate,setTimeout,setInterval,clearImmediate,clearTimeout,clearIntervalandref.

tty:onlyisatty(alwaysreturnsfalse).

vm:onlyrunInThisContext.

ThefollowingNode.jsmodulesarenotavailableatall:child_process,cluster,constants,crypto(butsee@arangodb/cryptobelow),dgram,dns,domain,http(butsee@arangodb/requestbelow),https,os,sys,tls,v8,zlib.

ArangoDBSpecificModulesTherearealargenumberofArangoDB-specificmodulesusingthe@arangodbnamespace,mostlyforinternalusebyArangoDBitself.Thefollowinghoweverarenoteworthy:

@arangodbprovidesdirectaccesstothedatabaseanditscollections.

@arangodb/cryptoprovidesvariouscryptographyfunctionsincludinghashingalgorithms.

JavaScriptModules

@arangodb/requestprovidesthefunctionalityformakingsynchronousHTTP/HTTPSrequests.

@arangodb/foxxisthenamespaceprovidingthevariousbuildingblocksoftheFoxxmicroserviceframework.

BundledNPMModules

ThefollowingNPMmodulesarepreinstalled:

aqbistheArangoDBQueryBuilderandcanbeusedtoconstructAQLquerieswithachainingJavaScriptAPI.

chaiisafull-featuredassertionlibraryforwritingJavaScripttests.

dedentisasimpleutilityfunctionforformattingmulti-linestrings.

error-stack-parserparsesstacktracesintoamoreusefulformat.

graphql-syncisanArangoDB-compatibleGraphQLserver/schemaimplementation.

highlight.jsisanHTMLsyntaxhighlighter.

i(inflect)isautilitylibraryforinflecting(e.g.pluralizing)words.

iconv-liteisautilitylibraryforconvertingbetweencharacterencodings

joiisavalidationlibrarythatissupportedthroughouttheFoxxframework.

js-yamlisaJavaScriptimplementationoftheYAMLdataformat(apartialsupersetofJSON).

lodashisautilitybeltforJavaScriptprovidingvarioususefulhelperfunctions.

minimatchisaglobmatcherformatchingwildcardsinfilepaths.

qsprovidesutilitiesfordealingwithquerystringsusingadifferentformatthanthequerystringmodule.

semverisautilitylibraryforhandlingsemverversionnumbers.

sinonisamockinglibraryforwritingteststubs,mocksandspies.

timezoneisalibraryforconvertingdatetimevaluesbetweenformatsandtimezones.

JavaScriptModules

ArangoDBModuleconstarangodb=require('@arangodb')

Note:ThismoduleshouldnotbeconfusedwiththearangojsJavaScriptdriverwhichcanbeusedtoaccessArangoDBfromoutsidethedatabase.AlthoughtheAPIssharesimilaritiesandthefunctionalityoverlaps,thetwoarenotcompatiblewitheachotherandcannotbeusedinterchangeably.

The dbobjectarangodb.db

Thedbobjectrepresentsthecurrentdatabaseandletsyouaccesscollectionsandrunqueries.Formoreinformationseethedbobjectreference.

Examples

constthirteen=db._query('RETURN5+8').next();

The aqltemplatestringhandlerarangodb.aql

TheaqlfunctionisaJavaScripttemplatestringhandler.ItcanbeusedtowritecomplexAQLqueriesasmulti-linestringswithouthavingtoworryaboutbindVarsandthedistinctionbetweencollectionsandregularparameters.

TouseitjustprefixaJavaScripttemplatestring(theoneswithbackticksinsteadofquotes)withitsimportname(e.g.aql)andpassinvariableslikeyouwouldwitharegulartemplatestring.ThestringwillautomaticallybeconvertedintoanobjectwithqueryandbindVarsattributeswhichyoucanpassdirectlytodb._querytoexecute.Ifyoupassinacollectionitwillbeautomaticallyrecognizedasacollectionreferenceandhandledaccordingly.

TofindoutmoreaboutAQLseetheAQLdocumentation.

Examples

constfilterValue=23;

constmydata=db._collection('mydata');

constresult=db._query(aql`

FORdIN${mydata}

FILTERd.num>${filterValue}

RETURNd

`).toArray();

The errorsobjectarangodb.errors

ThisobjectprovidesusefulobjectsforeacherrorcodeArangoDBmightuseinArangoErrorerrors.ThisishelpfulwhentryingtocatchspecificerrorsraisedbyArangoDB,e.g.whentryingtoaccessadocumentthatdoesnotexist.EachobjecthasacodepropertycorrespondingtotheerrorNumfoundonArangoErrorerrors.

Foracompletelistoftheerrornamesandcodesyoumayencounterseetheappendixonerrorcodes.

Examples

consterrors=require('@arangodb').errors;

@arangodb

someCollection.document('does-not-exist');

}catch(e){

if(e.isArangoError&&e.errorNum===errors.ERROR_ARANGO_DOCUMENT_NOT_FOUND.code){

thrownewError('Documentdoesnotexist');

thrownewError('Somethingwentwrong');

The timefunctionarangodb.time

Thisfunctionprovidesthecurrenttimeinsecondsasafloatingpointvaluewithmicrosecondprecisison.

ThisfunctioncanbeusedinsteadofDate.now()whenadditionalprecisionisneeded.

Examples

consttime=require('@arangodb').time;

conststart=time();

db._query(someVerySlowQuery);

console.log(`Elapsedtime:${time()-start}secs`);

@arangodb

ConsoleModuleglobal.console===require('console')

Note:Youdon'tneedtoloadthismoduledirectly.TheconsoleobjectisgloballydefinedthroughoutArangoDBandprovidesaccesstoallfunctionsinthismodule.

console.assert

console.assert(expression,format,argument1,...)

Teststhatanexpressionistrue.Ifnot,logsamessageandthrowsanexception.

Examples

console.assert(value==="abc","expected:value===abc,actual:",value);

console.debug

console.debug(format,argument1,...)

Formatstheargumentsaccordingtoformatandlogstheresultasdebugmessage.Notethatdebugmessageswillonlybeloggediftheserverisstartedwithloglevelsdebugortrace.

Stringsubstitutionpatterns,whichcanbeusedinformat.

%%sstring%%d,%%iinteger%%ffloatingpointnumber%%oobjecthyperlink

Examples

console.debug("%s","thisisatest");

console.dir

console.dir(object)

Logsalistingofallpropertiesoftheobject.

Exampleusage:

console.dir(myObject);

console.errorconsole.error(format,argument1,...)

Formatstheargumentsaccordingto@FA{format}andlogstheresultaserrormessage.

Exampleusage:

console.error("error'%s':%s",type,message);

console

console.getline

console.getline()

Readsinalinefromtheconsoleandreturnsitasstring.

console.groupconsole.group(format,argument1,...)

Formatstheargumentsaccordingtoformatandlogstheresultaslogmessage.Opensanestedblocktoindentallfuturemessagessent.CallgroupEndtoclosetheblock.Representationofblockisuptotheplatform,itcanbeaninteractiveblockorjustasetofindentedsubmessages.

Exampleusage:

console.group("userattributes");

console.log("name",user.name);

console.log("id",user.id);

console.groupEnd();

console.groupCollapsedconsole.groupCollapsed(format,argument1,...)

Sameasconsole.group.

console.groupEnd

console.groupEnd()

Closesthemostrecentlyopenedblockcreatedbyacalltogroup.

console.infoconsole.info(format,argument1,...)

Formatstheargumentsaccordingtoformatandlogstheresultasinfomessage.

Exampleusage:

console.info("The%sjumpedover%dfences",animal,count);

console.log

console.log(format,argument1,...)

Formatstheargumentsaccordingtoformatandlogstheresultaslogmessage.Thisisanaliasforconsole.info.

console.timeconsole.time(name)

Createsanewtimerunderthegivenname.CalltimeEndwiththesamenametostopthetimerandlogthetimeelapsed.

Exampleusage:

console.time("mytimer");

console

console.timeEnd("mytimer");//thiswillprinttheelapsedtime

console.timeEndconsole.timeEnd(name)

Stopsatimercreatedbyacalltotimeandlogsthetimeelapsed.

console.timeEnd

console.trace()

LogsastacktraceofJavaScriptexecutionatthepointwhereitiscalled.

console.warnconsole.warn(format,argument1,...)

Formatstheargumentsaccordingtoformatandlogstheresultaswarnmessage.

console

CryptoModuleconstcrypto=require('@arangodb/crypto')

Thecryptomoduleprovidesimplementationsofvarioushashingalgorithmsaswellascryptographyrelatedfunctions.

Nonces

Thesefunctionsdealwithcryptographicnonces.

createNonce

crypto.createNonce():string

Createsacryptographicnonce.

Returnsthecreatednonce.

checkAndMarkNonce

crypto.checkAndMarkNonce(nonce):void

Checksandmarksanonce.

Arguments

nonce:string

Thenoncetocheckandmark.

Returnsnothing.

Randomvalues

Thefollowingfunctionsdealwithgeneratingrandomvalues.

crypto.rand():number

Generatesarandomintegerthatmaybepositive,negativeorevenzero.

Returnsthegeneratednumber.

genRandomAlphaNumbers

crypto.genRandomAlphaNumbers(length):string

Generatesastringofrandomalpabeticalcharactersanddigits.

Arguments

length:number

Thelengthofthestringtogenerate.

Returnsthegeneratedstring.

genRandomNumbers

crypto.genRandomNumbers(length):string

Generatesastringofrandomdigits.

crypto

Arguments

length:number

genRandomSalt

crypto.genRandomSalt(length):string

Generatesastringofrandom(printable)ASCIIcharacters.

Arguments

length:number

JSONWebTokens(JWT)

ThesemethodsimplementtheJSONWebTokenstandard.

jwtEncode

crypto.jwtEncode(key,message,algorithm):string

GeneratesaJSONWebTokenforthegivenmessage.

Arguments

key:string|null

Thesecretcryptographickeytobeusedtosignthemessageusingthegivenalgorithm.Notethatthisfunctionwillraiseanerrorifthekeyisomittedbutthealgorithmexpectsakey,andalsoifthealgorithmdoesnotexpectakeybutakeyisprovided(e.g.whenusing"none").

message:string

MessagetobeencodedasJWT.Notethatthemessagewillonlybebase64-encodedandsigned,notencrypted.Donotstoresensitiveinformationintokensunlesstheywillonlybehandledbytrustedparties.

algorithm:string

Nameofthealgorithmtouseforsigningthemessage,e.g."HS512".

ReturnstheJSONWebToken.

jwtDecode

crypto.jwtDecode(key,token,noVerify):string|null

Arguments

key:string|null

Thesecretcryptographickeythatwasusedtosignthemessageusingthealgorithmindicatedbythetoken.Notethatthisfunctionwillraiseanerrorifthekeyisomittedbutthealgorithmexpectsakey.

Ifthealgorithmdoesnotexpectakeybutakeyisprovided,thetokenwillfailtoverify.

token:string

Thetokentodecode.

crypto

Notethatthefunctionwillraiseanerrorifthetokenismalformed(e.g.doesnothaveexactlythreesegments).

noVerify:boolean(Default:false)

Whetherverificationshouldbeskipped.Ifthisissettotruethesignatureofthetokenwillnotbeverified.Otherwisethefunctionwillraiseanerrorifthesignaturecannotbeverifiedusingthegivenkey.

ReturnsthedecodedJSONmessageornullifnotokenisprovided.

jwtAlgorithms

AhelperobjectcontainingthesupportedJWTalgorithms.EachattributenamecorrespondstoaJWTalgandthevalueisanobjectwithsignandverifymethods.

jwtCanonicalAlgorithmName

crypto.jwtCanonicalAlgorithmName(name):string

AhelperfunctionthattranslatesaJWTalgvaluefoundinaJWTheaderintothecanonicalnameofthealgorithminjwtAlgorithms.Raisesanerrorifnoalgorithmwithamatchingnameisfound.

Arguments

name:string

Algorithmnametolookup.

Returnsthecanonicalnameforthealgorithm.

Hashingalgorithms

md5crypto.md5(message):string

HashesthegivenmessageusingtheMD5algorithm.

Arguments

message:string

Themessagetohash.

Returnsthecryptographichash.

crypto.sha1(message):string

HashesthegivenmessageusingtheSHA-1algorithm.

Arguments

message:string

Themessagetohash.

sha224

Arguments

crypto

message:string

Themessagetohash.

sha256crypto.sha256(message):string

Arguments

message:string

Themessagetohash.

sha384crypto.sha384(message):string

Arguments

message:string

Themessagetohash.

sha512

Arguments

message:string

Themessagetohash.

Miscellaneous

constantEqualscrypto.constantEquals(str1,str2):boolean

Comparestwostrings.Thisfunctioniteratesovertheentirelengthofbothstringsandcanhelpmakingcertaintimingattacksharder.

Arguments

str1:string

Thefirststringtocompare.

str2:string

Thesecondstringtocompare.

Returnstrueifthestringsareequal,falseotherwise.

pbkdf2

crypto

crypto.pbkdf2(salt,password,iterations,keyLength):string

GeneratesaPBKDF2-HMAC-SHA1hashofthegivenpassword.

Arguments

salt:string

Thecryptographicsalttohashthepasswordwith.

password:string

Themessageorpasswordtohash.

iterations:number

Thenumberofiterations.Thisshouldbeaveryhighnumber.OWASPrecommended64000iterationsin2012andrecommendsdoublingthatnumbereverytwoyears.

WhenusingPBKDF2forpasswordhashesitisalsorecommendedtoaddarandomvalue(typicallybetween0and32000)tothatnumberthatisdifferentforeachuser.

keyLength:number

Thekeylength.

crypto.hmac(key,message,algorithm):string

GeneratesanHMAChashofthegivenmessage.

Arguments

key:string

Thecryptographickeytousetohashthemessage.

message:string

Themessagetohash.

algorithm:string

Thenameofthealgorithmtouse.

crypto

FilesystemModuleconstfs=require('fs')

TheimplementationtriestofollowtheCommonJSFilesystem/A/0specificationwherepossible.

WorkingDirectory

Thedirectoryfunctionsbelowshouldn'tusethecurrentworkingdirectoryoftheserverlike.or./test.Youwillnotbeabletotellwhethertheenvironmenttheserverisrunninginwillpermitdirectorylisting,readingorwritingoffiles.

YoushouldeitherbaseyourdirectorieswithgetTempPath(),orasafoxxserviceusethemodule.context.basePath.

SingleFileDirectoryManipulation

exists

checksifafileofanytypeordirectoryexistsfs.exists(path)

Returnstrueifafile(ofanytype)oradirectoryexistsatagivenpath.Ifthefileisabrokensymboliclink,returnsfalse.

isFile

testsifpathisafilefs.isFile(path)

Returnstrueifthepathpointstoafile.

isDirectory

testsifpathisadirectoryfs.isDirectory(path)

Returnstrueifthepathpointstoadirectory.

getsthesizeofafilefs.size(path)

Returnsthesizeofthefilespecifiedbypath.

getsthelastmodificationtimeofafilefs.mtime(filename)

Returnsthelastmodificationdateofthespecifiedfile.ThedateisreturnedasaUnixtimestamp(numberofsecondselapsedsinceJanuary11970).

pathSeparatorfs.pathSeparator

Ifyouwanttocombinetwopathsyoucanusefs.pathSeparatorinsteadof/or\.

fs.join(path,filename)

Thefunctionreturnsthecombinationofthepathandfilename,e.g.fs.join(Hello/World,foo.bar)wouldreturnHello/World/foo.bar.

getTempFile

returnsthenamefora(new)temporaryfilefs.getTempFile(directory,createFile)

Returnsthenameforanewtemporaryfileindirectorydirectory.IfcreateFileistrue,anemptyfilewillbecreatedsonootherprocesscancreateafileofthesamename.

Note:Thedirectorydirectorymustexist.

getTempPath

returnsthetemporarydirectoryfs.getTempPath()

Returnstheabsolutepathofthetemporarydirectory

makeAbsolute

makesagivenpathabsolutefs.makeAbsolute(path)

Returnsthegivenstringifitisanabsolutepath,otherwiseanabsolutepathtothesamelocationisreturned.

setsfilepermissionsofspecifiedfiles(nonwindowsonly)fs.exists(path)

Returnstrueonsuccess.

returnsthedirectorylistingfs.list(path)

Thefunctionsreturnsthenamesofallthefilesinadirectory,inlexicallysortedorder.Throwsanexceptionifthedirectorycannotbetraversed(orpathisnotadirectory).

Note:thismeansthatlist("x")ofadirectorycontaining"a"and"b"wouldreturn["a","b"],not["x/a","x/b"].

listTree

returnsthedirectorytreefs.listTree(path)

Thefunctionreturnsanarraythatstartswiththegivenpath,andallofthepathsrelativetothegivenpath,discoveredbyadepthfirsttraversalofeverydirectoryinanyvisiteddirectory,reportingbutnottraversingsymboliclinkstodirectories.Thefirstpathisalways"" ,thepathrelativetoitself.

makeDirectory

createsadirectoryfs.makeDirectory(path)

Createsthedirectoryspecifiedbypath.

makeDirectoryRecursive

createsadirectoryfs.makeDirectoryRecursive(path)

Createsthedirectoryhierarchyspecifiedbypath.

remove

removesafilefs.remove(filename)

Removesthefilefilenameatthegivenpath.Throwsanexceptionifthepathcorrespondstoanythingthatisnotafileorasymboliclink.If"path"referstoasymboliclink,removesthesymboliclink.

removeDirectory

removesanemptydirectoryfs.removeDirectory(path)

Removesadirectoryifitisempty.Throwsanexceptionifthepathisnotanemptydirectory.

removeDirectoryRecursive

removesadirectoryfs.removeDirectoryRecursive(path)

Removesadirectorywithallsubelements.Throwsanexceptionifthepathisnotadirectory.

FileIO

readsinafilefs.read(filename)

Readsinafileandreturnsthecontentasstring.PleasenotethatthefilecontentmustbeencodedinUTF-8.

readsinafileasbase64fs.read64(filename)

Readsinafileandreturnsthecontentasstring.ThefilecontentisBase64encoded.

readBuffer

readsinafilefs.readBuffer(filename)

ReadsinafileandreturnsitscontentinaBufferobject.

readFileSyncfs.readFileSync(filename,encoding)

Readsthecontentsofthefilespecifiedinfilename.Ifencodingisspecified,thefilecontentswillbereturnedasastring.Supportedencodingsare:

utf8orutf-8ascii

base64

ucs2orucs-2utf16leorutf16behex

Ifnoencodingisspecified,thefilecontentswillbereturnedinaBufferobject.

fs.write(filename,content)

Writesthecontentintoafile.ContentcanbeastringoraBufferobject.Ifthefilealreadyexists,itistruncated.

writeFileSync

fs.writeFileSync(filename,content)

Thisisanaliasforfs.write(filename,content).

appendfs.append(filename,content)

Writesthecontentintoafile.ContentcanbeastringoraBufferobject.Ifthefilealreadyexists,thecontentisappendedattheend.

RecursiveManipulation

copyRecursive

copiesadirectorystructurefs.copyRecursive(source,destination)

Copiessourcetodestination.Exceptionswillbethrownon:

Failuretocopythefilespecifyingadirectoryfordestinationwhensourceisafilespecifyingadirectoryassourceanddestination

CopyFile

copiesafileintoatargetfilefs.copyFile(source,destination)

Copiessourcetodestination.IfDestinationisadirectory,afileofthesamenamewillbecreatedinthatdirectory,elsethecopywillgetthespecifiedfilename.

renamesafilefs.move(source,destination)

Movessourcetodestination.Failuretomovethefile,orspecifyingadirectoryfordestinationwhensourceisafilewillthrowanexception.Likewise,specifyingadirectoryassourceanddestinationwillfail.

unzipFile

unzipsafilefs.unzipFile(filename,outpath,skipPaths,overwrite,password)

Unzipsthezipfilespecifiedbyfilenameintothepathspecifiedbyoutpath.Overwritesanyexistingtargetfilesifoverwriteissettotrue.

Returnstrueifthefilewasunzippedsuccessfully.

zipFile

zipsafilefs.zipFile(filename,chdir,files,password)

Storesthefilesspecifiedbyfilesinthezipfilefilename.Ifthefilefilenamealreadyexists,anerroristhrown.Thelistofinputfilesfilesmustbegivenasalistofabsolutefilenames.Ifchdirisnotempty,thechdirprefixwillbestrippedfromthefilenameinthezipfile,sowhenitisunzippedfilenameswillberelative.Specifyingapasswordisoptional.

Returnstrueifthefilewaszippedsuccessfully.

RequestModuleconstrequest=require('@arangodb/request')

TherequestmoduleprovidesthefunctionalityformakingHTTPrequests.

MakingHTTPrequests

HTTPmethodhelpers

InadditiontotherequestfunctionconvenienceshorthandsareavailableforeachHTTPmethodintheformof,i.e.:

request.head(url,options)

request.get(url,options)

request.post(url,options)

request.put(url,options)

request.delete(url,options)

request.patch(url,options)

Theseareequivalenttousingtherequestfunctiondirectly,i.e.:

request[method](url,options)

//isequivalentto

request({method,url,...options});

Forexample:

request.get('http://localhost',{headers:{'x-session-id':'keyboardcat'}});

//isequivalentto

request({

method:'get',

url:'http://localhost',

headers:{'x-session-id':'keyboardcat'}

Therequestfunction

TherequestfunctioncanbeusedtomakeHTTPrequests.

request(options)

PerformsanHTTPrequestandreturnsaResponseobject.

Parameter

Therequestfunctiontakesthefollowingoptions:

urloruri:thefully-qualifiedURLoraparsedURLfromurl.parse.qs(optional):objectcontainingquerystringvaluestobeappendedtotheURL.useQuerystring:iftrue,usequerystringmoduletohandlequerystrings,otherwiseuseqsmodule.Default:false.method(optional):HTTPmethod(case-insensitive).Default:"GET".headers(optional):HTTPheaders(case-insensitive).Default:{}.body(optional):requestbody.MustbeastringorBuffer,oraJSONserializablevalueifjsonistrue.json:iftrue,bodywillbeserializedtoaJSONstringandtheContent-Typeheaderwillbesetto"application/json".AdditionallytheresponsebodywillalsobeparsedasJSON(unlessencodingissettonull).Default:false.form(optional):whensettoastringorobjectandnobodyhasbeenset,bodywillbesettoaquerystringrepresentationofthatvalueandtheContent-Typeheaderwillbesetto"application/x-www-form-urlencoded".AlsoseeuseQuerystring.auth(optional):anobjectwiththepropertiesusernameandpasswordforHTTPBasicauthenticationorthepropertybearerfor

request

HTTPBearertokenauthentication.

sslProtocol(optional):whichtlsversionshouldbeusedtoconnecttotheurl.Thedefaultis4whichisTLS1.0.Seesslprotocolformoreopitions.followRedirect:whetherHTTP3xxredirectsshouldbefollowed.Default:true.maxRedirects:themaximumnumberofredirectstofollow.Default:10.encoding:encodingtobeusedfortheresponsebody.Ifsettonull,theresponsebodywillbereturnedasaBuffer.Default:"utf-8".timeout:numberofsecondstowaitforaresponsebeforeabortingtherequest.returnBodyOnError:whethertheresponsebodyshouldbereturnedevenwhentheserverresponseindicatesanerror.Default:true.

ThefunctionreturnsaResponseobjectwiththefollowingproperties:

rawBody:therawresponsebodyasaBuffer.body:theparsedresponsebody.Ifencodingwasnotsettonull,thisisastring.Ifadditionallyjsonwassettotrueandtheresponsebodyiswell-formedJSON,thisistheparsedJSONdata.headers:anobjectcontainingtheresponseheaders.OtherwisethisisidenticaltorawBody.statusCodeandstatus:theHTTPstatuscodeoftheresponse,e.g.404.message:theHTTPstatusmessageoftheresponse,e.g.NotFound.

Therequestmodulesupportsapplication/x-www-form-urlencoded(URLencoded)formuploads:

varres=request.post('http://service.example/upload',{form:{key:'value'}});

varres=request.post({url:'http://service.example/upload',form:{key:'value'}});

varres=request({

method:'post',

url:'http://service.example/upload',

form:{key:'value'}

Formdatawillbeencodedusingtheqsmodulebydefault.

Ifyouwanttousethequerystringmoduleinstead,simplyusetheuseQuerystringoption.

IfyouwanttosubmitJSON-serializablevaluesasrequestbodies,justsetthejsonoption:

varres=request.post('http://service.example/notify',{body:{key:'value'},json:true});

varres=request.post({url:'http://service.example/notify',body:{key:'value'},json:true});

varres=request({

method:'post',

url:'http://service.example/notify',

body:{key:'value'},

json:true

HTTPauthentication

TherequestmodulesupportsbothHTTPBasicauthentication.Justpassthecredentialsviatheauthoption:

varres=request.get(

request

'http://service.example/secret',

{auth:{username:'jcd',password:'bionicman'}}

varres=request.get({

url:'http://service.example/secret',

auth:{username:'jcd',password:'bionicman'}

varres=request({

method:'get',

auth:{username:'jcd',password:'bionicman'}

AlternativelyyoucansupplythecredentialsviatheURL:

varusername='jcd';

varpassword='bionicman';

varres=request.get(

'http://'+

encodeURIComponent(username)+

encodeURIComponent(password)+

'@service.example/secret'

YoucanalsouseBearertokenauthentication:

varres=request.get(

'http://service.example/secret',

{auth:{bearer:'keyboardcat'}}

varres=request.get({

auth:{bearer:'keyboardcat'}

varres=request({

method:'get',

auth:{bearer:'keyboardcat'}

request

Module"actions"constactions=require('@arangodb/actions')

Theactionmoduleprovidestheinfrastructurefordefininglow-levelHTTPactions.

IfyouwanttodefineHTTPendpointsinArangoDByoushouldprobablyusetheFoxxmicroserviceframeworkinstead.

Basics

Errormessage

actions.getErrorMessage(code)

Returnstheerrormessageforanerrorcode.

StandardHTTPResultGeneratorsactions.defineHttp(options)

Definesanewaction.Theoptionsareasfollows:

options.url

TheURL,whichcanbeusedtoaccesstheaction.Thispathmightcontainslashes.Notethatthisactionwillalsobecalled,ifaurlisgivensuchthatoptions.urlisaprefixofthegivenurlandnolongerdefinitionmatches.

options.prefix

Iffalse,thenonlyusetheactionforexactmatches.Thedefaultistrue.

options.callback(request,response)

Therequestargumentcontainsadescriptionoftherequest.Arequestparameterfooisaccessibleasrequest.parametrs.foo.Arequestheaderbarisaccessibleasrequest.headers.bar.Assumethattheactionisdefinedfortheurl/foo/barandtherequesturlis/foo/bar/hugo/egon.Thenthesuffixparts["hugo","egon"]areavailibleinrequest.suffix.

Thecallbackmustdefinefilltheresponse.

response.responseCode:theresponsecoderesponse.contentType:thecontenttypeoftheresponseresponse.body:thebodyoftheresponse

YoucanusethefunctionsResultOkandResultErrortoeasilygeneratearesponse.

Resultok

actions.resultOk(req,res,code,result,headers)

Thefunctiondefinesaresponse.codeisthestatuscodetoreturn.resultistheresultobject,whichwillbereturnedasJSONobjectinthebody.headersisanarrayofheaderstoreturned.Thefunctionaddstheattributeerrorwithvaluefalseandcodewithvaluecodetotheresult.

Resultbad

actions.resultBad(req,res,error-code,msg,headers)

Thefunctiongeneratesanerrorresponse.

Resultnotfoundactions.resultNotFound(req,res,code,msg,headers)

actions

Resultunsupported

actions.resultUnsupported(req,res,headers)

Resulterror

actions.resultError(req,res,code,errorNum,errorMessage,headers,keyvals)*

Thefunctiongeneratesanerrorresponse.TheresponsebodyisanarraywithanattributeerrorMessagecontainingtheerrormessageerrorMessage,errorcontainingtrue,codecontainingcode,errorNumcontainingerrorNum,anderrorMessagecontainingtheerrormessageerrorMessage.keyvalsaremixedintotheresult.

ResultnotImplementedactions.resultNotImplemented(req,res,msg,headers)

Resultpermanentredirect

actions.resultPermanentRedirect(req,res,options,headers)

Thefunctiongeneratesaredirectresponse.

Resulttemporaryredirectactions.resultTemporaryRedirect(req,res,options,headers)

Thefunctiongeneratesaredirectresponse.

ArangoDBResultGenerators

Collectionnotfoundactions.collectionNotFound(req,res,collection,headers)

Indexnotfound

actions.indexNotFound(req,res,collection,index,headers)

Resultexceptionactions.resultException(req,res,err,headers,verbose)

Thefunctiongeneratesanerrorresponse.If@FA{verbose}issettotrueornotspecified(thedefault),thentheerrorstacktracewillbeincludedintheerrormessageifavailable.If@FA{verbose}isastringitwillbeprependedbeforetheerrormessageandthestacktracewillalsobeincluded.

actions

QueriesModuleconstqueries=require('@arangodb/aql/queries')

ThequerymoduleprovidestheinfrastructureforworkingwithcurrentlyrunningAQLqueriesviaarangosh.

Properties

queries.properties()Returnstheserverscurrentquerytrackingconfiguration;wechangetheslowquerythresholdtogetbetterresults:

arangosh>varqueries=require("@arangodb/aql/queries");

arangosh>queries.properties();

arangosh>queries.properties({slowQueryThreshold:1});

Currentlyrunningqueries

Wecreateataskthatspawnsqueries,sowehaveniceoutput.Sincethistaskusesresources,youmaywanttoincreaseperiod(andnotforgettoremoveit...afterwards):

arangosh>vartheQuery='FORsleepLoooongIN1..5LETsleepLoooonger=SLEEP(1000)RETURN

sleepLoooong';

arangosh>vartasks=require("@arangodb/tasks");

arangosh>tasks.register({

........>id:"mytask-1",

........>name:"thisisasampletasktospawnaslowaqlquery",

........>command:"require('@arangodb').db._query('"+theQuery+"');"

........>});

arangosh>queries.current();

showexecutionresultsThefunctionreturnsthecurrentlyrunningAQLqueriesasanarray.

SlowqueriesThefunctionreturnsthelastAQLqueriesthatexceededtheslowquerythresholdasanarray:

arangosh>queries.slow();

ClearslowqueriesClearthelistofslowAQLqueries:

arangosh>queries.clearSlow();

"code":200

arangosh>queries.slow();

queries

KillKillarunningAQLquery:

arangosh>varrunningQueries=queries.current().filter(function(query){

........>returnquery.query===theQuery;

........>});

arangosh>queries.kill(runningQueries[0].id);

"code":200

queries

Write-aheadlogconstwal=require('internal').wal

Thismoduleprovidesfunctionalityforadministeringthewrite-aheadlogs.Mostofthesefunctionsonlyreturnsensiblevalueswheninvokedwiththemmfilesenginebeingactive.

Configuration

retrievestheconfigurationofthewrite-aheadloginternal.wal.properties()

Retrievestheconfigurationofthewrite-aheadlog.TheresultisaJSONarraywiththefollowingattributes:

allowOversizeEntries:whetherornotoperationsthatarebiggerthanasinglelogfilecanbeexecutedandstoredlogfileSize:thesizeofeachwrite-aheadlogfilehistoricLogfiles:themaximumnumberofhistoriclogfilestokeepreserveLogfiles:themaximumnumberofreservelogfilesthatArangoDBallocatesinthebackgroundsyncInterval:theintervalforautomaticsynchronizationofnot-yetsynchronizedwrite-aheadlogdata(inmilliseconds)throttleWait:themaximumwaittimethatoperationswillwaitbeforetheygetabortedifcaseofwrite-throttling(inmilliseconds)throttleWhenPending:thenumberofunprocessedgarbage-collectionoperationsthat,whenreached,willactivatewrite-throttling.Avalueof0meansthatwrite-throttlingwillnotbetriggered.

Examples

arangosh>require("internal").wal.properties();

showexecutionresultsconfiguresthewrite-aheadloginternal.wal.properties(properties)

Configuresthebehaviorofthewrite-aheadlog.propertiesmustbeaJSONJSONobjectwiththefollowingattributes:

allowOversizeEntries:whetherornotoperationsthatarebiggerthanasinglelogfilecanbeexecutedandstoredlogfileSize:thesizeofeachwrite-aheadlogfilehistoricLogfiles:themaximumnumberofhistoriclogfilestokeepreserveLogfiles:themaximumnumberofreservelogfilesthatArangoDBallocatesinthebackgroundthrottleWait:themaximumwaittimethatoperationswillwaitbeforetheygetabortedifcaseofwrite-throttling(inmilliseconds)throttleWhenPending:thenumberofunprocessedgarbage-collectionoperationsthat,whenreached,willactivatewrite-throttling.Avalueof0meansthatwrite-throttlingwillnotbetriggered.

Specifyinganyoftheaboveattributesisoptional.Notspecifiedattributeswillbeignoredandtheconfigurationforthemwillnotbemodified.

Examples

arangosh>require("internal").wal.properties({

........>allowOverSizeEntries:true,

........>logfileSize:32*1024*1024});

Flushing

flushesthecurrentlyopenWALlogfileinternal.wal.flush(waitForSync,waitForCollector)

Flushesthewrite-aheadlog.Byflushingthecurrentlyactivewrite-aheadlogfile,thedatainitcanbetransferredtocollectionjournalsanddatafiles.Thisisusefultoensurethatalldataforacollectionispresentinthecollectionjournalsanddatafiles,forexample,whendumpingthedataofacollection.

Write-aheadlog

ThewaitForSyncoptiondetermineswhetherornottheoperationshouldblockuntilthenot-yetsynchronizeddatainthewrite-aheadlogwassynchronizedtodisk.

ThewaitForCollectoroperationcanbeusedtospecifythattheoperationshouldblockuntilthedataintheflushedloghasbeencollectedbythewrite-aheadloggarbagecollector.Notethatsettingthisoptiontotruemightblockforalongtimeiftherearelong-runningtransactionsandthewrite-aheadloggarbagecollectorcannotfinishgarbagecollection.

Examples

arangosh>require("internal").wal.flush();

Write-aheadlog

TaskManagementconsttasks=require('@arangodb/tasks')

Note:IfyouaretryingtoscheduletasksinFoxxyoushouldconsiderusingtheFoxxqueuesmoduleinstead,whichprovidesamorehigh-levelAPIthatalsopersiststasksacrossreboots.

IntroductiontoTaskManagementinArangoDB

ArangoDBcanexecuteuser-definedJavaScriptfunctionsasone-shotorperiodictasks.Thisfunctionalitycanbeusedtoimplementtimedorrecurringjobsinthedatabase.

TasksinArangoDBconsistofaJavaScriptsnippetorfunctionthatisexecutedwhenthetaskisscheduled.Ataskcanbeaone-shottask(meaningitisrunonceandnotrepeated)oraperiodictask(meaningthatitisre-scheduledaftereachexecution).Taskscanhaveoptionalparameters,whicharedefinedattasksetuptime.Theparametersspecifiedattasksetuptimewillbepassedasargumentstothetaskwheneveritgetsexecuted.PeriodicTaskshaveanexecutionfrequencythatneedstobespecifiedwhenthetaskissetup.One-shottaskshaveaconfigurabledelayafterwhichthey'llgetexecuted.

Taskswillbeexecutedontheservertheyhavebeensetupon.Taskswillnotbeshippedaroundinacluster.Ataskwillbeexecutedinthecontextofthedatabaseitwascreatedin.However,whendroppingadatabase,anytasksthatwerecreatedinthecontextofthisdatabasewillremainactive.Itisthereforesensibletofirstunregisterallactivetasksforadatabasebeforedroppingthedatabase.

TasksregisteredinArangoDBwillbeexecuteduntiltheservergetsshutdownorrestarted.Afterarestartoftheserver,anyuser-definedone-shotorperiodictaskswillbelost.

CommandsforWorkingwithTasksArangoDBprovidesthefollowingcommandsforworkingwithtasks.Allcommandscanbeaccessedviathetasksmodule,whichcanbeloadedlikethis:

require("@arangodb/tasks")

PleasenotethatthetasksmoduleisavailableinsidetheArangoDBserveronly.ItcannotbeusedfromtheArangoShellorArangoDB'swebinterface.

Registeratask

Toregisteratask,theJavaScriptsnippetorfunctionneedstobespecifiedinadditiontotheexecutionfrequency.Optionally,ataskcanhaveanidandaname.Ifnoidisspecified,itwillbeauto-assignedforanewtask.Thetaskidisalsothemeanstoaccessorunregisteratasklater.Tasknamesareinformationalonly.Theycanbeusedtomakeataskdistinguishablefromothertasksalsorunningontheserver.

Thefollowingserver-sidecommandsregisteratask.ThecommandtobeexecutedisaJavaScriptstringsnippetwhichprintsamessagetotheserver'slogfile:

consttasks=require("@arangodb/tasks");

tasks.register({

id:"mytask-1",

name:"thisisasnippettask",

period:15,

command:"require('console').log('hellofromsnippettask');"

Theabovehasregisterataskwithidmytask-1,whichwillbeexecutedevery15secondsontheserver.Thetaskwillwritealogmessagewheneveritisinvoked.

TaskscanalsobesetupusingaJavaScriptcallbackfunctionlikethis:

TaskManagement

tasks.register({

id:"mytask-2",

name:"thisisafunctiontask",

period:15,

command:function(){

require('console').log('hellofromfunctiontask');

Itisimportanttonotethatthecallbackfunctionislateboundandwillbeexecutedinadifferentcontextthaninthecreationcontext.Thecallbackfunctionmustthereforenotaccessanyvariablesdefinedoutsideofitsownscope.Thecallbackfunctioncanstilldefineanduseitsownvariables.

Topassparameterstoatask,theparamsattributecanbesetwhenregisteringatask.NotethattheparametersarelimitedtodatatypesusableinJSON(meaningnocallbackfunctionscanbepassedasparametersintoatask):

tasks.register({

id:"mytask-3",

name:"thisisaparametertask",

period:15,

vargreeting=params.greeting;

vardata=JSON.stringify(params.data);

require('console').log('%sfromparametertask:%s',greeting,data);

params:{greeting:"hi",data:"howareyou?"}

Registeringaone-shottaskworksthesameway,exceptthattheperiodattributemustbeomitted.Ifperiodisomitted,thenthetaskwillbeexecutedjustonce.Thetaskinvocationdelaycanoptionallybespecifiedwiththeoffsetattribute:

tasks.register({

id:"mytask-once",

name:"thisisaone-shottask",

offset:10,

require('console').log('youwillseemejustonce!');

Note:Whenspecifyinganoffsetvalueof0,ArangoDBwillinternallyaddaverysmallvaluetotheoffsetsowillbeslightlygreaterthanzero.

Unregisteratask

Afterataskhasbeenregistered,itcanbeunregisteredusingitsid:

tasks.unregister("mytask-1");

Notethatunregisteringanon-existingtaskwillthrowanexception.

ListalltasksTogetanoverviewofwhichtasksareregistered,thereisthegetmethod.Ifthegetmethodiscalledwithoutanyarguments,itwillreturnanarrayofalltasks:

TaskManagement

tasks.get();

Ifgetiscalledwithataskidargument,itwillreturninformationaboutthisparticulartask:

tasks.get("mytask-3");

Thecreatedattributeofataskrevealswhenataskwascreated.ItisreturnedasaUnixtimestamp.

TaskManagement

DeprecatedFeatureslistedinthissectionshouldnolongerbeused,becausetheyareconsideredobsoleteandmaygetremovedinafuturerelease.Theyarecurrentlykeptforbackwardcompatibility.Thereareusuallybetteralternativestoreplacetheoldfeatureswith:

SimpleQueries:Ideomaticinterfaceinarangoshtoperformtrivialqueries.TheyaresupersededbyAQLqueries,whichcanalsoberuninarangosh.AQLisalanguageonitsownandwaymorepowerfulthanSimpleQueriescouldeverbe.Infact,the(stillsupported)SimpleQueriesaretranslatedinternallytoAQL,thentheAQLqueryisoptimizedandrunagainstthedatabaseinrecentversions,becauseofbetterperformanceandreducedmaintenancecomplexity.

Actions:SnippetsofJavaScriptcodeontheserver-sideforminimalcustomendpoints.SincetheFoxxrevampin3.0,itbecamereallyeasytowriteFoxxMicroservices,whichallowyoutodefinecustomendpointsevenwithcomplexbusinesslogic.

Deprecated

SimpleQueries

ItisrecommendedtouseAQLinstead.

Simplequeriescanbeusedifthequeryconditionisstraightforward,i.e.,adocumentreference,alldocuments,aquery-by-example,orasimplegeoquery.Inasimplequeryyoucanspecifyexactlyonecollectionandonequerycriteria.InthefollowingsectionswedescribetheJavaScriptshellinterfaceforsimplequeries,whichyoucanusewithintheArangoDBshellandwithinactionsandtransactions.ForotherlanguagesseethecorrespondinglanguageAPIdocumentation.

YoucanfindalistofqueriesatCollectionMethods.

SimpleQueries

Pagination

ItisrecommendedtouseAQLinstead,seetheLIMIToperation.

If,forexample,youdisplaytheresultofausersearch,thenyouareingeneralnotinterestedinthecompletedresultset,butonlythefirst10orsodocuments.Ormaybethenext10documentsforthesecondpage.Inthiscase,youcantheskipandlimitoperators.TheseoperatorsworklikeLIMITinMySQL.

skipusedtogetherwithlimitcanbeusedtoimplementpagination.Theskipoperatorskipsoverthefirstndocuments.So,inordertocreateresultpageswith10resultdocumentsperpage,youcanuseskip(n*10).limit(10)toaccessthe10documentsonthenthpage.Thisresultshouldbesorted,sothatthepaginationworksinapredicableway.

limitquery.limit(number)

Limitsaresulttothefirstnumberdocuments.Specifyingalimitof0willreturnnodocumentsatall.Ifyoudonotneedalimit,justdonotaddthelimitoperator.Thelimitmustbenon-negative.

Ingeneraltheinputtolimitshouldbesorted.Otherwiseitwillbeunclearwhichdocumentswillbeincludedintheresultset.

Examples

arangosh>db.five.all().limit(2).toArray();

skipquery.skip(number)

Skipsthefirstnumberdocuments.Ifnumberispositive,thenthisnumberofdocumentsareskippedbeforereturningthequeryresults.

Ingeneraltheinputtoskipshouldbesorted.Otherwiseitwillbeunclearwhichdocumentswillbeincludedintheresultset.

Note:usingnegativeskipvaluesisdeprecatedasofArangoDB2.6andwillnotbesupportedinfutureversionsofArangoDB.

Examples

arangosh>db.five.all().skip(3).toArray();

showexecutionresultsIgnoreanylimitwithcount:

arangosh>db.five.all().limit(2).count();

Countinganylimitorskip:

arangosh>db.five.all().limit(2).count(true);

SimpleQueries

ModificationQueries

ItisrecommendedtouseAQLinstead,seeDataModificationQueries.

ArangoDBalsoallowsremoving,replacing,andupdatingdocumentsbasedonanexampledocument.Everydocumentinthecollectionwillbecomparedagainstthespecifiedexampledocumentandbedeleted/replaced/updatedifallattributesmatch.

Thesemethodshouldbeusedwithcautionastheyareintendedtoremoveormodifylotsofdocumentsinacollection.

Allmethodscanoptionallyberestrictedtoaspecificnumberofoperations.However,ifalimitisspecificbutislessthanthenumberofmatches,itwillbeundefinedwhichofthematchingdocumentswillgetremoved/modified.RemovebyExample,ReplacebyExampleandUpdatebyExamplearedescribedwithexamplesinthesubchapterCollectionMethods.

SimpleQueries

GeoQueries

ItisrecommendedtouseAQLinstead,seeGeofunctions.

TheArangoDBallowstoselectdocumentsbasedongeographiccoordinates.Inorderforthistowork,ageo-spatialindexmustbedefined.ThisindexwilluseaveryelaboratealgorithmtolookupneighborsthatisamagnitudefasterthanasimpleR*index.

Ingeneralageocoordinateisapairoflatitudeandlongitude,whichmustbothbespecifiedasnumbers.Ageoindexcanbecreatedoncoordinatesthatarestoredinasinglelistattributewithtwoelementslike[-10,+30](latitudefirst,followedbylongitude)oroncoordinatesstoredintwoseparateattributes.

Forexample,toindexthefollowingdocuments,anindexcanbecreatedonthepositionattributeofthedocuments:

db.test.save({position:[-10,30]});

db.test.save({position:[10,45.5]});

db.test.ensureIndex({type:"geo",fields:["position"]});

Ifcoordinatesarestoredintwodistinctattributes,theindexmustbecreatedonthetwoattributes:

db.test.save({latitude:-10,longitude:30});

db.test.save({latitude:10,longitude:45.5});

db.test.ensureIndex({type:"geo",fields:["latitude","longitude"]});

Inordertofindalldocumentswithinagivenradiusaroundacoordinateusethewithinoperator.Inordertofindalldocumentsnearagivendocumentusethenearoperator.

Itispossibletodefinemorethanonegeo-spatialindexpercollection.Inthiscaseyoumustgiveahintusingthegeooperatorwhichofindexesshouldbeusedinaquery.

constructsanearqueryforacollectioncollection.near(latitude,longitude)Thereturnedlistissortedaccordingtothedistance,withthenearestdocumenttothecoordinate(latitude,longitude)comingfirst.Ifthereareneardocumentsofequaldistance,documentsarechosenrandomlyfromthissetuntilthelimitisreached.Itispossibletochangethelimitusingthelimitoperator.Inordertousethenearoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesforthedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.Note:neardoesnotsupportnegativeskips.//However,youcanstilluselimitfollowedtoskip.collection.near(latitude,longitude).limit(limit)Limitstheresulttolimitdocumentsinsteadofthedefault100.Note:Unlikewithmultipleexplicitlimits,limitwillraisetheimplicitdefaultlimitimposedbywithin.collection.near(latitude,longitude).distance()Thiswilladdanattributedistancetoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.collection.near(latitude,longitude).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thenearsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionusingthenearoperatoristousetheAQLNEARfunctioninanAQLqueryasfollows:

RETURNdoc

Examples

Togetthenearesttwolocations:

SimpleQueries

........>for(varj=-180;j<=180;j+=10){

........>name:"Name/"+i+"/"+j,

........>loc:[i,j]});

........>}}

showexecutionresultsIfyouneedthedistanceaswell,thenyoucanusethedistanceoperator:

........>for(varj=-180;j<=180;j+=10){

........>name:"Name/"+i+"/"+j,

........>loc:[i,j]});

........>}}

arangosh>db.geo.near(0,0).distance().limit(2).toArray();

Within

constructsawithinqueryforacollectioncollection.within(latitude,longitude,radius)Thiswillfindalldocumentswithinagivenradiusaroundthecoordinate(latitude,longitude).Thereturnedarrayissortedbydistance,beginningwiththenearestdocument.Inordertousethewithinoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesforthedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.collection.within(latitude,longitude,radius).distance()Thiswilladdanattribute_distancetoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.collection.within(latitude,longitude,radius).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thewithinsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionusingthewithinoperatoristousetheAQLWITHINfunctioninanAQLqueryasfollows:

RETURNdoc

Examples

Tofindalldocumentswithinaradiusof2000kmuse:

........>for(varj=-180;j<=180;j+=10){

........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});}}

arangosh>db.geo.within(0,0,2000*1000).distance().toArray();

constructsageoindexselectioncollection.geo(location-attribute)Looksupageoindexdefinedonattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Thisisusefulforcollectionswithmultipledefinedgeoindexes.collection.geo(location_attribute,true)Looksupageoindexonacompoundattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.collection.geo(latitude_attribute,longitude_attribute)

SimpleQueries

Looksupageoindexdefinedonthetwoattributeslatitude_attributeandlongitude-attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thegeosimplequeryhelperfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforrunninggeoqueriesistousetheirAQLequivalents.

Examples

Assumeyouhavealocationstoredaslistintheattributehomeandadestinationstoredintheattributework.Thenyoucanusethegeooperatortoselectwhichgeo-spatialattributes(andthuswhichindex)touseinanearquery.

........>for(j=-180;j<=180;j+=10){

........>db.complex.save({name:"Name/"+i+"/"+j,

........>home:[i,j],

........>work:[-i,-j]});

........>}

........>

arangosh>db.complex.near(0,170).limit(5);

arangosh>db.complex.ensureIndex({type:"geo",fields:["home"]});

arangosh>db.complex.near(0,170).limit(5).toArray();

arangosh>db.complex.geo("work").near(0,170).limit(5);

arangosh>db.complex.ensureIndex({type:"geo",fields:["work"]});

arangosh>db.complex.geo("work").near(0,170).limit(5).toArray();

Relatedtopics

OtherArangoDBgeographicfeaturesaredescribedin:

AQLGeofunctionsGeoindexes

SimpleQueries

Fulltextqueries

ItisrecommendedtouseAQLinstead,seeFulltextfunctions.

ArangoDBallowstorunqueriesontextcontainedindocumentattributes.Tousethis,afulltextindexmustbedefinedfortheattributeofthecollectionthatcontainsthetext.Creatingtheindexwillparsethetextinthespecifiedattributeforalldocumentsofthecollection.Onlydocumentswillbeindexedthatcontainatextualvalueintheindexedattribute.Forsuchdocuments,thetextvaluewillbeparsed,andtheindividualwordswillbeinsertedintothefulltextindex.

Whenafulltextindexexists,itcanbequeriedusingafulltextquery.

Fulltext

queriesthefulltextindexcollection.fulltext(attribute,query)

Thefulltextsimplequeryfunctionsperformsafulltextsearchonthespecifiedattributeandthespecifiedquery.

Detailsaboutthefulltextquerysyntaxcanbefoundbelow.

Note:thefulltextsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforexecutingfulltextqueriesistouseanAQLqueryusingtheFULLTEXTAQLfunctionasfollows:

FORdocINFULLTEXT(@@collection,@attributeName,@queryString,@limit)

RETURNdoc

Examples

arangosh>db.emails.ensureFulltextIndex("content");

arangosh>db.emails.save({content:

........>"HelloAlice,howareyoudoing?Regards,Bob"});

arangosh>db.emails.save({content:

........>"HelloCharlie,doAliceandBobknowaboutit?"});

arangosh>db.emails.save({content:"Ithinktheydon'tknow.Regards,Eve"});

arangosh>db.emails.fulltext("content","charlie,|eve").toArray();

FulltextSyntax:

Inthesimplestform,afulltextquerycontainsjustthesoughtword.Ifmultiplesearchwordsaregiveninaquery,theyshouldbeseparatedbycommas.AllsearchwordswillbecombinedwithalogicalANDbydefault,andonlysuchdocumentswillbereturnedthatcontainallsearchwords.Thisdefaultbehaviorcanbechangedbyprovidingtheextracontrolcharactersinthefulltextquery,whichare:

+:logicalAND(intersection)|:logicalOR(union)-:negation(exclusion)

Examples:

"banana":searchesfordocumentscontaining"banana""banana,apple":searchesfordocumentscontainingboth"banana"AND"apple""banana,|orange":searchesfordocumentscontainingeither"banana"OR"orange"ORboth"banana,-apple":searchesfordocumentsthatcontains"banana"butNOT"apple".

Logicaloperatorsareevaluatedfromlefttoright.

SimpleQueries

Eachsearchwordcanoptionallybeprefixedwithcomplete:orprefix:,withcomplete:beingthedefault.Thisallowssearchingforcompletewordsorforwordprefixes.Suffixsearchesoranyotherformsarepartial-wordmatchingarecurrentlynotsupported.

Examples:

"complete:banana":searchesfordocumentscontainingtheexactword"banana""prefix:head":searchesfordocumentswithwordsthatstartwithprefix"head""prefix:head,banana":searchesfordocumentscontainwordsstartingwithprefix"head"andthatalsocontaintheexactword"banana".

Completematchandprefixsearchoptionscanbecombinedwiththelogicaloperators.

Pleasenotethatonlywordswithaminimumlengthwillgetindexed.Thisminimumlengthcanbedefinedwhencreatingthefulltextindex.Forwordstokenization,thelibicutextboundaryanalysisisused,whichtakesintoaccountthedefaultasdefinedatserverstartup(--server.default-languagestartupoption).Generally,thewordboundaryanalysiswillfilteroutpunctuationbutwillnotdomuchmore.

Especiallynowordnormalization,stemming,orsimilarityanalysiswillbeperformedwhenindexingorsearching.Ifanyofthesefeaturesisrequired,itissuggestedthattheuserdoesthetextnormalizationontheclientside,andprovidesforeachdocumentanextraattributecontainingjustacomma-separatedlistofnormalizedwords.Thisattributecanthenbeindexedwithafulltextindex,andtheusercansendfulltextqueriesforthisindex,withthefulltextqueriesalsocontainingthestemmedornormalizedversionsofwordsasrequiredbytheuser.

SimpleQueries

ArangoDB'sActions

ItisrecommendedtouseFoxxinstead.

IntroductiontoUserActions

InsomewaysthecommunicationlayeroftheArangoDBserverbehaveslikeaWebserver.UnlikeaWebserver,itnormallyrespondstoHTTPrequestsbydeliveringJSONobjects.Remember,documentsinthedatabasearejustJSONobjects.So,mostofthetimetheHTTPresponsewillcontainaJSONdocumentfromthedatabaseasbody.YoucanextractthedocumentsstoredinthedatabaseusingHTTPGET.YoucanstoredocumentsusingHTTPPOST.

However,thereissomethingmore.Youcanwritesmallsnippets-socalledactions-toextendthedatabase.TheideaofactionsisthatsometimesitisbettertostorepartsofthebusinesslogicwithinArangoDB.

Thesimplestexampleistheageofaperson.Assumeyoustoreinformationaboutpeopleinyourdatabase.Itisananti-patterntostoretheage,becauseitchangeseverynowandthen.Therefore,younormallystorethebirthdayandlettheclientdecidewhattodowithit.However,ifyouhavemanydifferentclients,itmightbeeasiertoenrichthepersondocumentwiththeageusingactionsonceontheserverside.

Or,forinstance,ifyouwanttoapplysomestatisticstolargedata-setsandyoucannoteasilyexpressthisasquery.Youcandefineaactioninsteadoftransferringthewholedatatotheclientanddothecomputationontheclient.

Actionsarealsousefulifyouwanttorestrictandfilterdataaccordingtosomecomplexpermissionsystem.

TheArangoDBservercandeliverallkindsofinformation,JSONbeingonlyonepossibleformat.YoucanalsogenerateHTMLorimages.However,aWebserverisnormallybettersuitedforthetaskasitalsoimplementsvariouscachingstrategies,languageselection,compressionandsoon.Havingsaidthat,therearestillsituationswhereitmightbesuitabletousetheArangoDBtodeliverHTMLpages-staticordynamic.Asimpleexampleisthebuilt-inadministrationinterface.YoucanaccessitusinganymodernbrowserandthereisnoneedforaseparateApacheorIIS.

IngeneralyouwilluseFoxxtoeasilyextendthedatabasewithbusinesslogic.Foxxprovidesansimpletouseinterfacetoactions.

Thefollowingsectionswillexplainthelow-levelactionswithinArangoDBonwhichFoxxisbuiltandshowhowtodefinethem.TheexamplesstartwithdeliveringstaticHTMLpages-evenifthisisnottheprimaryuse-caseforactions.ThelatersectionswillthenshowyouhowtocodesomepiecesofyourbusinesslogicandreturnJSONobjects.

TheinterfaceislooselymodeledaftertheJavaScriptclassesforHTTPrequestandresponsesfoundinnode.jsandthemiddleware/routingaspectsofconnect.jsandexpress.js.

Notethatunlikenode.js,ArangoDBismulti-threadedandthereisnoeasywaytosharestatebetweenqueriesinsidetheJavaScriptengine.Ifsuchstateinformationisrequired,youneedtousethedatabaseitself.

Actions

AHelloWorldExample

TheclientAPIorbrowsersendsaHTTPrequesttotheArangoDBserverandtheserverreturnsaHTTPresponsetotheclient.AHTTPrequestconsistsofamethod,normallyGETorPOSTwhenusingabrowser,andarequestpathlike/hello/world.ForarealWebserverthereareazillionofotherthingtoconsider,wewillignorethisforthemoment.TheHTTPresponsecontainsacontenttype,describinghowtointerpretthereturneddata,andthedataitself.

Inthefollowingexample,wewanttodefineanactioninArangoDB,sothattheserverreturnstheHTMLdocument

<html>

<body>

HelloWorld

</body>

</html>

ifaskedGET/hello/world.

Theserverneedstoknowwhatfunctiontocallorwhatdocumenttodeliverifitreceivesarequest.Thisiscalledrouting.AlltheroutinginformationofArangoDBisstoredinacollection_routing.Eachentryinthiscollectionsdescribeshowtodealwithaparticularrequestpath.

Fortheaboveexample,addthefollowingdocumenttothe_routingcollection:

arangosh>db._routing.save({

........>url:{

........>match:"/hello/world"

........>},

........>content:{

........>contentType:"text/html",

........>body:"<html><body>HelloWorld</body></html>"

........>}

........>});

showexecutionresultsInordertoactivatethenewrouting,youmusteitherrestarttheserverorcalltheinternalreloadfunction.

arangosh>require("internal").reloadRouting()

Nowusethebrowserandaccesshttp://localhost:8529/hello/world

YoushouldseetheHelloWorldinourbrowser:

shell>curl--dump-http://localhost:8529/hello/world

HTTP/1.1200OK

content-type:text/html

x-content-type-options:nosniff

"HelloWorld"

MatchingaURLTherearealotofoptionsfortheurlattribute.Ifyoudefinedifferentroutingforthesamepath,thenthefollowingsimpleruleisappliedinordertodeterminewhichmatchwins:Iftherearetwomatches,thenthemorespecificwins.I.e,ifthereisawildcardmatchandanexactmatch,theexactmatchispreferred.Ifthereisashortandalongmatch,thelongermatchwins.

Actions

ExactMatch

Ifthedefinitionis

match:"/hello/world"

thenthematchmustbeexact.Onlytherequestfor/hello/worldwillmatch,everythingelse,e.g./hello/world/myor/hello/world2,willnotmatch.

Thefollowingdefinitionisashort-cutforanexactmatch.

url:"/hello/world"

Note:WhilethetwodefinitionswillresultinthesameURLmatching,thereisasubtledifferencebetweenthem:

Theformerdefinition(definingurlasanobjectwithamatchattribute)willresultintheURLbeingaccessibleviaallsupportedHTTPmethods(e.g.GET,POST,PUT,DELETE,...),whereasthelatterdefinition(providingastringurlattribute)willresultintheURLbeingaccessibleviaHTTPGETandHTTPHEADonly,withallotherHTTPmethodsbeingdisabled.CallingaURLwithanunsupportedordisabledHTTPmethodwillresultinanHTTP501(notimplemented)error.

PrefixMatch

Ifthedefinitionis

match:"/hello/world/*"

thenthematchcanbeaprefixmatch.Therequestsfor/hello/world,/hello/world/my,and/hello/world/how/are/youwillallmatch.However/hello/world2doesnotmatch.PrefixmatcheswithinaURLpart,i.e./hello/world*,arenotallowed.Thewildcardmustoccurattheend,i.e.

/hello/*/world

isalsodisallowed.

Ifyoudefinetworoutes

{url:{match:"/hello/world/*"}}

{url:{match:"/hello/world/emil"}}

thenthesecondroutewillbeusedfor/hello/world/emilbecauseitismorespecific.

ParameterizedMatch

Aparameterizedmatchissimilartoaprefixmatch,buttheparametersarealsoallowedinsidetheURLpath.

Ifthedefinitionis

match:"/hello/:name/world"

Actions

thentheURLmusthavethreeparts,thefirstpartbeinghelloandthethirdpartworld.Forexample,/hello/emil/worldwillmatch,while/hello/emil/meyer/worldwillnot.

ConstraintMatch

Aconstraintmatchissimilartoaparameterizedmatch,buttheparameterscancarryconstraints.

Ifthedefinitionis

match:"/hello/:name/world",

constraint:{

name:"/[a-z]+/"

thentheURLmusthavethreeparts,thefirstpartbeinghelloandthethirdpartworld.Thesecondpartmustbealllowercase.

ItispossibletousemorethenoneconstraintforthesameURLpart.

match:"/hello/:name|:id/world",

constraint:{

name:"/[a-z]+/",id:"/[0-9]+/"

OptionalMatch

Anoptionalmatchissimilartoaparameterizedmatch,butthelastparameterisoptional.

Ifthedefinitionis

match:"/hello/:name?",

constraint:{

name:"/[a-z]+/"

thentheURL/helloand/hello/emilwillmatch.

Ifthedefinitionsare

{url:{match:"/hello/world"}}

{url:{match:"/hello/:name",constraint:{name:"/[a-z]+/"}}}

{url:{match:"/hello/*"}}

thentheURL/hello/worldwillbematchedbythefirstroute,becauseitisthemostspecific.TheURL/hello/youwillbematchedbythesecondroute,becauseitismorespecificthantheprefixmatch.

MethodRestriction

YoucanrestrictthematchtospecificHTTPmethods.

Ifthedefinitionis

Actions

match:"/hello/world",

methods:["post","put"]

thenonlyHTTPPOSTandPUTrequestswillmatch.CallingwithadifferentHTTPmethodwillresultinanHTTP501error.

Pleasenotethatifurlisdefinedasasimplestring,thenonlytheHTTPmethodsGETandHEADwillbeallowed,anallothermethodswillbedisabled:

url:"/hello/world"

MoreonMatching

Rememberthatthemorespecificmatchwins.

Amatchwithoutparameterorwildcardismorespecificthanamatchwithparametersorwildcard.Amatchwithparameterismorespecificthanamatchwithawildcard.Ifthereismorethanoneparameter,specificityisappliedfromlefttoright.

Considerthefollowingdefinitions

........>url:{match:"/hello/world"},

........>content:{contentType:"text/plain",body:"MatchNo1"}});

........>url:{match:"/hello/:name",constraint:{name:"/[a-z]+/"}},

........>url:{match:"/:something/world"},

........>url:{match:"/hi/*"},

showexecutionresultsThen

HTTP/1.1200OK

content-type:text/plain

"MatchNo1"

shell>curl--dump-http://localhost:8529/hello/emil

HTTP/1.1200OK

"MatchNo2"

shell>curl--dump-http://localhost:8529/your/world

HTTP/1.1200OK

Actions

"MatchNo3"

shell>curl--dump-http://localhost:8529/hi/you

HTTP/1.1200OK

"MatchNo4"

Youcanwritethefollowingdocumentintothe_routingcollectiontotesttheaboveexamples.

routes:[

{url:{match:"/hello/world"},content:"route1"},

{url:{match:"/hello/:name|:id",constraint:{name:"/[a-z]+/",id:"/[0-9]+/"}},content:"route2"},

{url:{match:"/:something/world"},content:"route3"},

{url:{match:"/hello/*"},content:"route4"},

Actions

AHelloWorldExampleforJSON

Ifyouchangetheexampleslightly,thenaJSONobjectwillbedelivered.

........>url:"/hello/json",

........>content:{

........>contentType:"application/json",

........>body:'{"hello":"world"}'

........>}

........>});

showexecutionresultsAgaincheckwithyourbrowserorcURLhttp://localhost:8529/hello/json

Dependingonyourbrowserandinstalledadd-onsyouwilleitherseetheJSONobjectoradownloaddialog.IfyourbrowserwantstoopenanexternalapplicationtodisplaytheJSONobject,youcanchangethecontentTypeto"text/plain" fortheexample.Thismakesiteasiertochecktheexampleusingabrowser.Orusecurltoaccesstheserver.

shell>curl--dump-http://localhost:8529/hello/json

HTTP/1.1200OK

content-type:application/json;charset=utf-8

"hello":"world"

DeliveringContent

Therearealotofdifferentwaysonhowtodelivercontent.Wehavealreadyseenthesimplestone,wherestaticcontentisdelivered.Thefun,however,startswhendeliveringdynamiccontent.

StaticContent

Youcanspecifyabodyandacontent-type.

........>url:"/hello/contentType",

........>content:{

........>contentType:"text/html",

........>body:"<html><body>HelloWorld</body></html>"

........>}

........>});

shell>curl--dump-http://localhost:8529/hello/contentType

Actions

HTTP/1.1200OK

"HelloWorld"

Ifthecontenttypeistext/plainthenyoucanusetheshort-cut

content:"HelloWorld"

ASimpleAction

Thesimplestdynamicactionis:

action:{

do:"@arangodb/actions/echoRequest"

Itisnotadvisabletostorefunctionsdirectlyintheroutingtable.Itisbettertocallfunctionsdefinedinmodules.IntheaboveexamplethefunctioncanbeaccessedfromJavaScriptas:

require("@arangodb/actions").echoRequest

ThefunctionechoRequestispre-defined.Ittakestherequestobjectsandechositintheresponse.

Thesignatureofsuchafunctionmustbe

function(req,res,options,next)

Examples

........>url:"/hello/echo",

........>action:{

........>do:"@arangodb/actions/echoRequest"

........>}

........>});

showexecutionresultsReloadtheroutingandcheckhttp://127.0.0.1:8529/hello/echo

Youshouldseesomethinglike

arangosh>arango.GET("/hello/echo")

Therequestmightcontainpath,prefix,suffix,andurlParametersattributes.pathisthecompletepathassuppliedbytheuserandalwaysavailable.Ifaprefixwasmatched,thenthisprefixisstoredintheattributeprefixandtheremainingURLpartsarestoredasanarrayinsuffix.Ifoneormoreparameterswerematched,thentheparametervaluesarestoredinurlParameters.

Actions

Forexample,iftheurldescriptionis

match:"/hello/:name/:action"

andyourequestthepath/hello/emil/jump,thentherequestobjectwillcontainthefollowingattribute

urlParameters:{

name:"emil",

action:"jump"

ActionController

Asanalternativetothesimpleaction,youcanusecontrollers.Acontrollerisamodule,definesthefunctionget,put,post,delete,head,patch.Ifarequestofthecorrespondingtypeismatched,thefunctionwillbecalled.

Examples

........>action:{

........>controller:"@arangodb/actions/echoController"

........>}

........>});

showexecutionresultsReloadtheroutingandcheckhttp://127.0.0.1:8529/hello/echo:

PrefixActionController

Thecontrollerisselectedwhenthedefinitionisread.Thereisamoreflexible,butslowerandmaybeinsecurevariant,theprefixcontroller.

Assumethattheurlisaprefixmatch

match:/hello/*"

Youcanuse

action:{

prefixController:"@arangodb/actions"

todefineaprefixcontroller.IftheURL/hello/echoControllerisgiven,thenthemodule@arangodb/actions/echoControllerisused.

Ifyouuseaprefixcontroller,youshouldmakecertainthatnounwantedactionsareavailableundertheprefix.

Actions

Thedefinition

action:"@arangodb/actions"

isashort-cutforaprefixcontrollerdefinition.

FunctionAction

Youcanalsostoreafunctiondirectlyintheroutingtable.

Examples

........>action:{

........>callback:"function(req,res){res.statusCode=200;res.body='Hello'}"

........>}

........>});

arangosh>arango.GET("hello/echo")

arangosh>db._query("FORrouteIN_routingFILTERroute.url=='/hello/echo'REMOVEroute

in_routing")

[objectArangoQueryCursor,count:0,cached:false,hasMore:false]

RequestsandResponses

Thecontrollermustdefinehandlerfunctionswhichtakearequestobjectandfilltheresponseobject.

AverysimpleexampleisthefunctionechoRequestdefinedinthemodule@arangodb/actions.

function(req,res,options,next){

varresult;

result={request:req,options:options};

res.responseCode=exports.HTTP_OK;

res.contentType="application/json";

res.body=JSON.stringify(result);

Installitvia:

........>url:"/echo",

........>action:{

........>}

........>})

showexecutionresultsReloadtheroutingandcheckhttp://127.0.0.1:8529/hello/echo

Youshouldseesomethinglike

Actions

arangosh>db._query("FORrouteIN_routingFILTERroute.url=='/hello/echo'REMOVEroute

in_routing")

showexecutionresultsYoumayalsopassoptionstothecalledfunction:

........>url:"/echo",

........>action:{

........>do:"@arangodb/actions/echoRequest",

........>options:{

........>"Hello":"World"

........>}

........>});

showexecutionresultsYounowseetheoptionsintheresult:

arangosh>arango.GET("/echo")

arangosh>db._query("FORrouteIN_routingFILTERroute.url=='/echo'REMOVEroutein

_routing")

Actions

ModifyingRequestandResponse

Aswe'veseeninthepreviousexamples,actionsgetcalledwiththerequestandresponseobjects(namedreqandresintheexamples)passedasparameterstotheirhandlerfunctions.

ThereqobjectcontainstheincomingHTTPrequest,whichmightormightnothavebeenmodifiedbyapreviousaction(ifactionswerechained).

Ahandlercanmodifytherequestobjectinplaceifdesired.Thismightbeusefulwhenwritingmiddleware(seebelow)thatisusedtointerceptincomingrequests,modifythemandpassthemtotheactualhandlers.

Whilemodifyingtherequestobjectmightnotbethatrelevantfornon-middlewareactions,modifyingtheresponseobjectdefinitelyis.Modifyingtheresponseobjectisanaction'sonlywaytoreturndatatothecalleroftheaction.

We'vealreadyseenhowtosettheHTTPstatuscode,thecontenttype,andtheresultbody.Theresobjecthasthefollowingpropertiesforthese:

contentType:MIMEtypeofthebodyasdefinedintheHTTPstandard(e.g.text/html,text/plain,application/json,...)responsecode:theHTTPstatuscodeoftheresponseasdefinedintheHTTPstandard.Commonvaluesforactionsthatsucceedare200or201.PleaserefertotheHTTPstandardformoreinformation.body:theactualresponsedata

Tosetormodifyarbitraryheadersoftheresponseobject,theheaderspropertycanbeused.Forexample,toaddauser-definedheadertotheresponse,thefollowingcodewilldo:

res.headers=res.headers||{};//headersmightormightnotbepresent

res.headers['X-Test']='someValue';//setheaderX-Testto"someValue"

ThiswillsettheadditionalHTTPheaderX-TesttovaluesomeValue.Otherheaderscanbesetaswell.NotethatArangoDBmightchangethecaseoftheheadernamestolowercasewhenassemblingtheoverallresponsethatissenttothecaller.

ItisnotnecessarytoexplicitlysetaContent-LengthheaderfortheresponseasArangoDBwillcalculatethecontentlengthautomaticallyandaddthisheaderitself.ArangoDBmightalsoaddaConnectionheaderitselftohandleHTTPkeep-alive.

ArangoDBalsosupportsautomatictransformationofthebodydatatoanotherformat.Currently,theonlysupportedtransformationsarebase64-encodingandbase64-decoding.Usingthetransformations,anactioncancreateabase64encodedbodyandstillletArangoDBsendthenon-encodedversion,forexample:

res.body='VGhpcyBpcyBhIHRlc3Q=';

res.transformations=res.transformations||[];//initialize

res.transformations.push('base64decode');//willbase64decodetheresponsebody

WhenArangoDBprocessestheresponse,itwillbase64-decodewhat'sinres.bodyandsettheHTTPheaderContent-Encoding:binary.Theoppositecanbeachievedwiththebase64encodetransformation:ArangoDBwillthenautomaticallybase64-encodethebodyandsetaContent-Encoding:base64HTTPheader.

WritingdynamicactionhandlersTowriteyourowndynamicactionhandlers,youmustputthemintomodules.

Modulesareameansoforganizingactionhandlersandmakingthemloadableunderspecificnames.

Tostart,we'lldefineasimpleactionhandlerinamodule/ownTest:

arangosh>db._modules.save({

........>path:"/db:/ownTest",

........>content:

........>"exports.do=function(req,res,options,next){"+

........>"res.body='test';"+

Actions

........>"res.responseCode=200;"+

........>"res.contentType='text/plain';"+

........>"};"

........>});

showexecutionresultsThisdoesnothingbutregisteradoactionhandlerinamodule/ownTest.Theactionhandlerisnotyetcallable,butmustbemappedtoaroutefirst.Tomaptheactiontotheroute/ourtest,executethefollowingcommand:

........>url:"/ourtest",

........>action:{

........>controller:"db://ownTest"

........>}

........>});

showexecutionresultsNowusethebrowserorcURLandaccesshttp://localhost:8529/ourtest:

shell>curl--dump-http://localhost:8529/ourtest

HTTP/1.1200OK

"test"

Youwillseethatthemodule'sdofunctionhasbeenexecuted.

AWordaboutCachingSometimesitmightseemthatyourchangedonottakeeffect.Inthiscasetheculpritcouldbetheroutingcaches:

Theroutingcachestorestheroutinginformationcomputedfromthe_routingcollection.Wheneveryouchangethiscollectionmanually,youneedtocall

inordertorebuildthecache.

AdvancedUsagesFordetailedinformationseethereferencemanual.

Redirects

Usethefollowingforapermanentredirect:

........>url:"/redirectMe",

........>action:{

........>do:"@arangodb/actions/redirectRequest",

........>options:{

Actions

........>permanently:true,

........>destination:"/somewhere.else/"

........>}

........>});

shell>curl--dump-http://localhost:8529/redirectMe

HTTP/1.1301MovedPermanently

location:/somewhere.else/

"<html><head><title>Moved</title></head><body><h1>Moved</h1><p>Thispagehasmovedto<a

href=\"/somewhere.else/\">/somewhere.else/</a>.</p></body></html>"

RoutingBundles

Insteadofaddingallroutesforpackageseparately,youcanspecifyabundle:

........>routes:[

........>{

........>url:"/url1",

........>content:"route1"

........>},

........>{

........>url:"/url2",

........>},

........>{

........>url:"/url3",

........>}

........>]

........>});

shell>curl--dump-http://localhost:8529/url2

HTTP/1.1200OK

"route2"

shell>curl--dump-http://localhost:8529/url3

HTTP/1.1200OK

Actions

"route3"

Theadvantageis,thatyoucanputallyourroutesintoonedocumentanduseacommonprefix.

........>urlPrefix:"/test",

........>routes:[

........>{

........>url:"/url1",

........>},

........>{

........>url:"/url2",

........>},

........>{

........>url:"/url3",

........>}

........>]

........>});

showexecutionresultswilldefinetheURL/test/url1,/test/url2,and/test/url3:

shell>curl--dump-http://localhost:8529/test/url1

HTTP/1.1200OK

"route1"

HTTP/1.1200OK

"route2"

HTTP/1.1200OK

"route3"

WritingMiddleware

Assume,youwanttologeveryrequestinyournamespacetotheconsole.(ifArangoDBisrunningasadaemon,thiswillendupinthelogfile).InthiscaseyoucaneasilydefineanactionfortheURL/subdirectory.Thisactionsimplylogstherequests,callsthenextinline,andlogstheresponse:

arangosh>db._modules.save({

Actions

........>path:"/db:/OwnMiddlewareTest",

........>content:

........>"exports.logRequest=function(req,res,options,next){"+

........>"console=require('console');"+

........>"console.log('receivedrequest:%s',JSON.stringify(req));"+

........>"next();"+

........>"console.log('producedresponse:%s',JSON.stringify(res));"+

........>"};"

........>});

showexecutionresultsThisfunctionwillnowbeavailableasdb://OwnMiddlewareTest/logRequest.YouneedtotellArangoDBthatitisshoulduseaprefixmatchandthattheshortestmatchshouldwininthiscase:

........>middleware:[

........>{

........>url:{

........>match:"/subdirectory/*"

........>},

........>action:{

........>do:"db://OwnMiddlewareTest/logRequest"

........>}

........>]

........>});

showexecutionresultsWhenyoucallnext()inthataction,thenextspecificroutingwillbeusedfortheoriginalURL.EvenifyoumodifytheURLintherequestobjectreq,thiswillnotcausethenext()tojumptotheroutingdefinedforthisnextURL.IfproceedsoccurringtheoriginURL.However,ifyouusenext(true),theroutingwillstopandrequesthandlingisstartedwiththenewURL.Youmustensurethatnext(true)isnevercalledwithoutmodifyingtheURLintherequestobjectreq.Otherwiseanendlessloopwilloccur.

Nowweaddsomeothersimpleroutingstotestallthis:

........>url:"/subdirectory/ourtest/1",

........>action:{

........>}

........>});

........>action:{

........>}

........>});

........>action:{

........>}

........>});

Actions

Thenwesendsomecurlrequeststothesesampleroutes:

shell>curl--dump-http://localhost:8529/subdirectory/ourtest/1

HTTP/1.1200OK

showresponsebodyandtheconsole(and/orthelogfile)willshowrequestsandreplies.Notethatloggingdoesn'twarrantthesequenceinwhichtheselineswillappear.

ApplicationDeploymentUsingsingleroutesorbundlescanbebecomeabitmessyinlargeapplications.Kaerushaswrittenadeploymenttoolinnode.js.

NotethatthereisalsoFoxxforbuildingapplicationswithArangoDB.

CommonPitfallswhenusingActions

Caching

IfyoumadeanychangestotheroutingbutthechangesdoesnothaveanyeffectwhencallingthemodifiedactionsURL,youmighthavebeenhitbysomecachingissues.

Afteranymodificationtotheroutingoractions,itisthusrecommendedtomakethechanges"live"bycallingthefollowingfunctionsfromwithinarangosh:

Youmightalsobeaffectedbyclient-sidecaching.BrowserstendtocachecontentandalsoredirectionURLs.Youmightneedtoclearordisablethebrowsercacheinsomecasestoseeyourchangesineffect.

Datatypes

Whenprocessingtherequestdatainanaction,pleasebeawarethatthedatatypeofallqueryparametersisstring.ThisisbecausethewholeURLisastringandwhentheindividualpartsareextracted,theywillalsobestrings.

Forexample,whencallingtheURLhttp://localhost:8529/hello/world?value=5

theparametervaluewillhaveavalueof(string)5,not(number)5.ThismightbetroublesomeifyouuseJavaScript's===operatorwhencheckingrequestparametervalues.

ThesameproblemoccurswithincomingHTTPheaders.WhensendingthefollowingheaderfromaclienttoArangoDB

X-My-Value:5

thentheheaderX-My-Valuewillhaveavalueof(string)5andnot(number)5.

404NotFound

IfyoudefinedaURLintheroutingandtheURLisaccessiblefineviaHTTPGETbutreturnsanHTTP501(notimplemented)forotherHTTPmethodssuchasPOST,PUTorDELETE,thenyoumighthavebeenhitbysomedefaults.

Bydefault,URLsdefinedlikethis(simplestringurlattribute)areaccessibleviaHTTPGETandHEADonly.TomakesuchURLsaccessibleviaotherHTTPmethods,extendtheURLdefinitionwiththemethodsattribute.

Forexample,thisdefinitiononlyallowsaccessviaGETandHEAD:

Actions

url:"/hello/world"

whereasthisdefinitionallowsHTTPGET,POST,andPUT:

........>url:{

........>match:"/hello/world",

........>methods:["get","post","put"]

........>},

........>action:{

........>}

........>});

HTTP/1.1200OK

showresponsebodyTheformerdefinition(definingurlasanobjectwithamatchattribute)willresultintheURLbeingaccessibleviaallsupportedHTTPmethods(e.g.GET,POST,PUT,DELETE,...),whereasthelatterdefinition(providingastringurlattribute)willresultintheURLbeingaccessibleviaHTTPGETandHTTPHEADonly,withallotherHTTPmethodsbeingdisabled.CallingaURLwithanunsupportedordisabledHTTPmethodwillresultinanHTTP404error.

Actions

Generalerrors

0-ERROR_NO_ERRORNoerrorhasoccurred.

1-ERROR_FAILEDWillberaisedwhenageneralerroroccurred.

2-ERROR_SYS_ERRORWillberaisedwhenoperatingsystemerroroccurred.

3-ERROR_OUT_OF_MEMORYWillberaisedwhenthereisamemoryshortage.

4-ERROR_INTERNALWillberaisedwhenaninternalerroroccurred.

5-ERROR_ILLEGAL_NUMBERWillberaisedwhenanillegalrepresentationofanumberwasgiven.

6-ERROR_NUMERIC_OVERFLOWWillberaisedwhenanumericoverflowoccurred.

7-ERROR_ILLEGAL_OPTIONWillberaisedwhenanunknownoptionwassuppliedbytheuser.

8-ERROR_DEAD_PIDWillberaisedwhenaPIDwithoutalivingprocesswasfound.

9-ERROR_NOT_IMPLEMENTEDWillberaisedwhenhittinganunimplementedfeature.

10-ERROR_BAD_PARAMETERWillberaisedwhentheparameterdoesnotfulfilltherequirements.

11-ERROR_FORBIDDENWillberaisedwhenyouaremissingpermissionfortheoperation.

12-ERROR_OUT_OF_MEMORY_MMAPWillberaisedwhenthereisamemoryshortage.

13-ERROR_CORRUPTED_CSVWillberaisedwhenencounteringacorruptcsvline.

14-ERROR_FILE_NOT_FOUNDWillberaisedwhenafileisnotfound.

15-ERROR_CANNOT_WRITE_FILEWillberaisedwhenafilecannotbewritten.

16-ERROR_CANNOT_OVERWRITE_FILEWillberaisedwhenanattemptismadetooverwriteanexistingfile.

17-ERROR_TYPE_ERRORWillberaisedwhenatypeerrorisunencountered.

18-ERROR_LOCK_TIMEOUTWillberaisedwhenthere'satimeoutwaitingforalock.

19-ERROR_CANNOT_CREATE_DIRECTORYWillberaisedwhenanattempttocreateadirectoryfails.

20-ERROR_CANNOT_CREATE_TEMP_FILEWillberaisedwhenanattempttocreateatemporaryfilefails.

21-ERROR_REQUEST_CANCELEDWillberaisedwhenarequestiscanceledbytheuser.

22-ERROR_DEBUGWillberaisedintentionallyduringdebugging.

25-ERROR_IP_ADDRESS_INVALIDWillberaisedwhenthestructureofanIPaddressisinvalid.

27-ERROR_FILE_EXISTSWillberaisedwhenafilealreadyexists.

28-ERROR_LOCKEDWillberaisedwhenaresourceoranoperationislocked.

29-ERROR_DEADLOCKWillberaisedwhenadeadlockisdetectedwhenaccessingcollections.

30-ERROR_SHUTTING_DOWNWillberaisedwhenacallcannotsucceedbecauseaservershutdownisalreadyinprogress.

31-ERROR_ONLY_ENTERPRISEWillberaisedwhenanenterprise-featureisrequestedfromthecommunityedition.

32-ERROR_RESOURCE_LIMITWillberaisedwhentheresourcesusedbyanoperationexceedtheconfiguredmaximumvalue.

HTTPerrorstatuscodes400-ERROR_HTTP_BAD_PARAMETERWillberaisedwhentheHTTPrequestdoesnotfulfilltherequirements.

401-ERROR_HTTP_UNAUTHORIZEDWillberaisedwhenauthorizationisrequiredbuttheuserisnotauthorized.

403-ERROR_HTTP_FORBIDDENWillberaisedwhentheoperationisforbidden.

404-ERROR_HTTP_NOT_FOUNDWillberaisedwhenanURIisunknown.

405-ERROR_HTTP_METHOD_NOT_ALLOWEDWillberaisedwhenanunsupportedHTTPmethodisusedforanoperation.

406-ERROR_HTTP_NOT_ACCEPTABLEWillberaisedwhenanunsupportedHTTPcontenttypeisusedforanoperation,orifarequestisnotacceptableforaleaderorfollower.

412-ERROR_HTTP_PRECONDITION_FAILEDWillberaisedwhenapreconditionforanHTTPrequestisnotmet.

500-ERROR_HTTP_SERVER_ERRORWillberaisedwhenaninternalserverisencountered.

503-ERROR_HTTP_SERVICE_UNAVAILABLEWillberaisedwhenaserviceistemporarilyunavailable.

HTTPprocessingerrors

600-ERROR_HTTP_CORRUPTED_JSONWillberaisedwhenastringrepresentationofaJSONobjectiscorrupt.

601-ERROR_HTTP_SUPERFLUOUS_SUFFICESWillberaisedwhentheURLcontainssuperfluoussuffices.

InternalArangoDBstorageerrors

Forerrorsthatoccurbecauseofaprogrammingerror.

1000-ERROR_ARANGO_ILLEGAL_STATEInternalerrorthatwillberaisedwhenthedatafileisnotintherequiredstate.

1002-ERROR_ARANGO_DATAFILE_SEALEDInternalerrorthatwillberaisedwhentryingtowritetoadatafile.

1004-ERROR_ARANGO_READ_ONLYInternalerrorthatwillberaisedwhentryingtowritetoaread-onlydatafileorcollection.

1005-ERROR_ARANGO_DUPLICATE_IDENTIFIERInternalerrorthatwillberaisedwhenaidentifierduplicateisdetected.

1006-ERROR_ARANGO_DATAFILE_UNREADABLEInternalerrorthatwillberaisedwhenadatafileisunreadable.

1007-ERROR_ARANGO_DATAFILE_EMPTYInternalerrorthatwillberaisedwhenadatafileisempty.

1008-ERROR_ARANGO_RECOVERYWillberaisedwhenanerroroccurredduringWALlogfilerecovery.

1009-ERROR_ARANGO_DATAFILE_STATISTICS_NOT_FOUNDWillberaisedwhenarequireddatafilestatisticsobjectwasnotfound.

ExternalArangoDBstorageerrorsForerrorsthatoccurbecauseofanoutsideevent.

1100-ERROR_ARANGO_CORRUPTED_DATAFILEWillberaisedwhenacorruptionisdetectedinadatafile.

1101-ERROR_ARANGO_ILLEGAL_PARAMETER_FILEWillberaisedifaparameterfileiscorruptedorcannotberead.

1102-ERROR_ARANGO_CORRUPTED_COLLECTIONWillberaisedwhenacollectioncontainsoneormorecorrupteddatafiles.

1103-ERROR_ARANGO_MMAP_FAILEDWillberaisedwhenthesystemcallmmapfailed.

1104-ERROR_ARANGO_FILESYSTEM_FULLWillberaisedwhenthefilesystemisfull.

1105-ERROR_ARANGO_NO_JOURNALWillberaisedwhenajournalcannotbecreated.

1106-ERROR_ARANGO_DATAFILE_ALREADY_EXISTSWillberaisedwhenthedatafilecannotbecreatedorrenamedbecauseafileofthesamenamealreadyexists.

1107-ERROR_ARANGO_DATADIR_LOCKEDWillberaisedwhenthedatabasedirectoryislockedbyadifferentprocess.

1108-ERROR_ARANGO_COLLECTION_DIRECTORY_ALREADY_EXISTSWillberaisedwhenthecollectioncannotbecreatedbecauseadirectoryofthesamenamealreadyexists.

1109-ERROR_ARANGO_MSYNC_FAILEDWillberaisedwhenthesystemcallmsyncfailed.

1110-ERROR_ARANGO_DATADIR_UNLOCKABLEWillberaisedwhentheservercannotlockthedatabasedirectoryonstartup.

1111-ERROR_ARANGO_SYNC_TIMEOUTWillberaisedwhentheserverwaitedtoolongforadatafiletobesyncedtodisk.

GeneralArangoDBstorageerrorsForerrorsthatoccurwhenfulfillingauserrequest.

1200-ERROR_ARANGO_CONFLICTWillberaisedwhenupdatingordeletingadocumentandaconflicthasbeendetected.

1201-ERROR_ARANGO_DATADIR_INVALIDWillberaisedwhenanon-existingdatabasedirectorywasspecifiedwhenstartingthedatabase.

1202-ERROR_ARANGO_DOCUMENT_NOT_FOUNDWillberaisedwhenadocumentwithagivenidentifierorhandleisunknown.

1203-ERROR_ARANGO_COLLECTION_NOT_FOUNDWillberaisedwhenacollectionwiththegivenidentifierornameisunknown.

1204-ERROR_ARANGO_COLLECTION_PARAMETER_MISSINGWillberaisedwhenthecollectionparameterismissing.

1205-ERROR_ARANGO_DOCUMENT_HANDLE_BADWillberaisedwhenadocumenthandleiscorrupt.

1206-ERROR_ARANGO_MAXIMAL_SIZE_TOO_SMALLWillberaisedwhenthemaximalsizeofthejournalistoosmall.

1207-ERROR_ARANGO_DUPLICATE_NAMEWillberaisedwhenanameduplicateisdetected.

1208-ERROR_ARANGO_ILLEGAL_NAMEWillberaisedwhenanillegalnameisdetected.

1209-ERROR_ARANGO_NO_INDEXWillberaisedwhennosuitableindexforthequeryisknown.

1210-ERROR_ARANGO_UNIQUE_CONSTRAINT_VIOLATEDWillberaisedwhenthereisauniqueconstraintviolation.

1211-ERROR_ARANGO_VIEW_NOT_FOUNDWillberaisedwhenaviewwiththegivenidentifierornameisunknown.

1212-ERROR_ARANGO_INDEX_NOT_FOUNDWillberaisedwhenanindexwithagivenidentifierisunknown.

1213-ERROR_ARANGO_CROSS_COLLECTION_REQUESTWillberaisedwhenacross-collectionisrequested.

1214-ERROR_ARANGO_INDEX_HANDLE_BADWillberaisedwhenaindexhandleiscorrupt.

1216-ERROR_ARANGO_DOCUMENT_TOO_LARGEWillberaisedwhenthedocumentcannotfitintoanydatafilebecauseofitistoolarge.

1217-ERROR_ARANGO_COLLECTION_NOT_UNLOADEDWillberaisedwhenacollectionshouldbeunloaded,buthasadifferentstatus.

1218-ERROR_ARANGO_COLLECTION_TYPE_INVALIDWillberaisedwhenaninvalidcollectiontypeisusedinarequest.

1219-ERROR_ARANGO_VALIDATION_FAILEDWillberaisedwhenthevalidationofanattributeofastructurefailed.

1220-ERROR_ARANGO_ATTRIBUTE_PARSER_FAILEDWillberaisedwhenparsinganattributenamedefinitionfailed.

1221-ERROR_ARANGO_DOCUMENT_KEY_BADWillberaisedwhenadocumentkeyiscorrupt.

1222-ERROR_ARANGO_DOCUMENT_KEY_UNEXPECTEDWillberaisedwhenauser-defineddocumentkeyissuppliedforcollectionswithautokeygeneration.

1224-ERROR_ARANGO_DATADIR_NOT_WRITABLEWillberaisedwhentheserver'sdatabasedirectoryisnotwritableforthecurrentuser.

1225-ERROR_ARANGO_OUT_OF_KEYSWillberaisedwhenakeygeneratorrunsoutofkeys.

1226-ERROR_ARANGO_DOCUMENT_KEY_MISSINGWillberaisedwhenadocumentkeyismissing.

1227-ERROR_ARANGO_DOCUMENT_TYPE_INVALIDWillberaisedwhenthereisanattempttocreateadocumentwithaninvalidtype.

1228-ERROR_ARANGO_DATABASE_NOT_FOUNDWillberaisedwhenanon-existingdatabaseisaccessed.

1229-ERROR_ARANGO_DATABASE_NAME_INVALIDWillberaisedwhenaninvaliddatabasenameisused.

1230-ERROR_ARANGO_USE_SYSTEM_DATABASEWillberaisedwhenanoperationisrequestedinadatabaseotherthanthesystemdatabase.

1231-ERROR_ARANGO_ENDPOINT_NOT_FOUNDWillberaisedwhenthereisanattempttodeleteanon-existingendpoint.

1232-ERROR_ARANGO_INVALID_KEY_GENERATORWillberaisedwhenaninvalidkeygeneratordescriptionisused.

1233-ERROR_ARANGO_INVALID_EDGE_ATTRIBUTEwillberaisedwhenthe_fromor_tovaluesofanedgeareundefinedorcontainaninvalidvalue.

1234-ERROR_ARANGO_INDEX_DOCUMENT_ATTRIBUTE_MISSINGWillberaisedwhenanattempttoinsertadocumentintoanindexiscausedbyinthedocumentnothavingoneormoreattributeswhichtheindexisbuilton.

1235-ERROR_ARANGO_INDEX_CREATION_FAILEDWillberaisedwhenanattempttocreateanindexhasfailed.

1236-ERROR_ARANGO_WRITE_THROTTLE_TIMEOUTWillberaisedwhentheserveriswrite-throttledandawriteoperationhaswaitedtoolongfortheservertoprocessqueuedoperations.

1237-ERROR_ARANGO_COLLECTION_TYPE_MISMATCHWillberaisedwhenacollectionhasadifferenttypefromwhathasbeenexpected.

1238-ERROR_ARANGO_COLLECTION_NOT_LOADEDWillberaisedwhenacollectionisaccessedthatisnotyetloaded.

1239-ERROR_ARANGO_DOCUMENT_REV_BADWillberaisedwhenadocumentrevisioniscorruptorismissingwhereneeded.

CheckedArangoDBstorageerrors

Forerrorsthatoccurbutareanticipated.

1300-ERROR_ARANGO_DATAFILE_FULLWillberaisedwhenthedatafilereachesitslimit.

1301-ERROR_ARANGO_EMPTY_DATADIRWillberaisedwhenencounteringanemptyserverdatabasedirectory.

1302-ERROR_ARANGO_TRY_AGAINWillberaisedwhenanoperationshouldberetried.

1303-ERROR_ARANGO_BUSYWillberaisedwhenstorageengineisbusy.

1304-ERROR_ARANGO_MERGE_IN_PROGRESSWillberaisedwhenstorageenginehasadatafilemergeinprogressandcannotcompletetheoperation.

1305-ERROR_ARANGO_IO_ERRORWillberaisedwhenstorageengineencountersanI/Oerror.

ArangoDBreplicationerrors1400-ERROR_REPLICATION_NO_RESPONSEWillberaisedwhenthereplicationapplierdoesnotreceiveanyoranincompleteresponsefromthemaster.

1401-ERROR_REPLICATION_INVALID_RESPONSEWillberaisedwhenthereplicationapplierreceivesaninvalidresponsefromthemaster.

1402-ERROR_REPLICATION_MASTER_ERRORWillberaisedwhenthereplicationapplierreceivesaservererrorfromthemaster.

1403-ERROR_REPLICATION_MASTER_INCOMPATIBLEWillberaisedwhenthereplicationapplierconnectstoamasterthathasanincompatibleversion.

1404-ERROR_REPLICATION_MASTER_CHANGEWillberaisedwhenthereplicationapplierconnectstoadifferentmasterthanbefore.

1405-ERROR_REPLICATION_LOOPWillberaisedwhenthereplicationapplierisaskedtoconnecttoitselfforreplication.

1406-ERROR_REPLICATION_UNEXPECTED_MARKERWillberaisedwhenanunexpectedmarkerisfoundinthereplicationlogstream.

1407-ERROR_REPLICATION_INVALID_APPLIER_STATEWillberaisedwhenaninvalidreplicationapplierstatefileisfound.

1408-ERROR_REPLICATION_UNEXPECTED_TRANSACTIONWillberaisedwhenanunexpectedtransactionidisfound.

1410-ERROR_REPLICATION_INVALID_APPLIER_CONFIGURATIONWillberaisedwhentheconfigurationforthereplicationapplierisinvalid.

1411-ERROR_REPLICATION_RUNNINGWillberaisedwhenthereisanattempttoperformanoperationwhilethereplicationapplierisrunning.

1412-ERROR_REPLICATION_APPLIER_STOPPEDSpecialerrorcodeusedtoindicatethereplicationapplierwasstoppedbyauser.

1413-ERROR_REPLICATION_NO_START_TICKWillberaisedwhenthereplicationapplierisstartedwithoutaknownstarttickvalue.

1414-ERROR_REPLICATION_START_TICK_NOT_PRESENTWillberaisedwhenthereplicationapplierfetchesdatausingastarttick,butthatstarttickisnotpresentontheloggerserveranymore.

ERROR_REPLICATION_WRONG_CHECKSUM_FORMAT,1415,"thechecksumformatiswrong","Willberaisedwhentheformatofthechecksumiswrong")

1416-ERROR_REPLICATION_WRONG_CHECKSUMWillberaisedwhenanewbornfollowersubmitsawrongchecksum

1417-ERROR_REPLICATION_SHARD_NONEMPTYWillberaisedwhenashardisnotemptyandthefollowertriesashortcut

ArangoDBclustererrors

1450-ERROR_CLUSTER_NO_AGENCYWillberaisedwhennoneoftheagencyserverscanbeconnectedto.

1451-ERROR_CLUSTER_NO_COORDINATOR_HEADERWillberaisedwhenaDBserverinaclusterreceivesaHTTPrequestwithoutacoordinatorheader.

1452-ERROR_CLUSTER_COULD_NOT_LOCK_PLANWillberaisedwhenacoordinatorinaclustercannotlockthePlanhierarchyintheagency.

1453-ERROR_CLUSTER_COLLECTION_ID_EXISTSWillberaisedwhenacoordinatorinaclustertriestocreateacollectionandthecollectionIDalreadyexists.

1454-ERROR_CLUSTER_COULD_NOT_CREATE_COLLECTION_IN_PLANWillberaisedwhenacoordinatorinaclustercannotcreateanentryforanewcollectioninthePlanhierarchyintheagency.

1455-ERROR_CLUSTER_COULD_NOT_READ_CURRENT_VERSIONWillberaisedwhenacoordinatorinaclustercannotreadtheVersionentryintheCurrenthierarchyintheagency.

1456-ERROR_CLUSTER_COULD_NOT_CREATE_COLLECTIONWillberaisedwhenacoordinatorinaclusternoticesthatsomeDBServersreportproblemswhencreatingshardsforanewcollection.

1457-ERROR_CLUSTER_TIMEOUTWillberaisedwhenacoordinatorinaclusterrunsintoatimeoutforsomeclusterwideoperation.

1458-ERROR_CLUSTER_COULD_NOT_REMOVE_COLLECTION_IN_PLANWillberaisedwhenacoordinatorinaclustercannotremoveanentryforacollectioninthePlanhierarchyintheagency.

1459-ERROR_CLUSTER_COULD_NOT_REMOVE_COLLECTION_IN_CURRENTWillberaisedwhenacoordinatorinaclustercannotremoveanentryforacollectionintheCurrenthierarchyintheagency.

1460-ERROR_CLUSTER_COULD_NOT_CREATE_DATABASE_IN_PLANWillberaisedwhenacoordinatorinaclustercannotcreateanentryforanewdatabaseinthePlanhierarchyintheagency.

1461-ERROR_CLUSTER_COULD_NOT_CREATE_DATABASEWillberaisedwhenacoordinatorinaclusternoticesthatsomeDBServersreportproblemswhencreatingdatabasesforanewclusterwidedatabase.

1462-ERROR_CLUSTER_COULD_NOT_REMOVE_DATABASE_IN_PLANWillberaisedwhenacoordinatorinaclustercannotremoveanentryforadatabaseinthePlanhierarchyintheagency.

1463-ERROR_CLUSTER_COULD_NOT_REMOVE_DATABASE_IN_CURRENTWillberaisedwhenacoordinatorinaclustercannotremoveanentryforadatabaseintheCurrenthierarchyintheagency.

1464-ERROR_CLUSTER_SHARD_GONEWillberaisedwhenacoordinatorinaclustercannotdeterminetheshardthatisresponsibleforagivendocument.

1465-ERROR_CLUSTER_CONNECTION_LOSTWillberaisedwhenacoordinatorinaclusterlosesanHTTPconnectiontoaDBserverintheclusterwhilsttransferringdata.

1466-ERROR_CLUSTER_MUST_NOT_SPECIFY_KEYWillberaisedwhenacoordinatorinaclusterfindsthatthe_keyattributewasspecifiedinashardedcollectiontheusesnotonly_keyasshardingattribute.

1467-ERROR_CLUSTER_GOT_CONTRADICTING_ANSWERSWillberaisedifacoordinatorinaclustergetsconflictingresultsfromdifferentshards,whichshouldneverhappen.

1468-ERROR_CLUSTER_NOT_ALL_SHARDING_ATTRIBUTES_GIVENWillberaisedifacoordinatortriestofindoutwhichshardisresponsibleforapartialdocument,butcannotdothisbecausenotallshardingattributesarespecified.

1469-ERROR_CLUSTER_MUST_NOT_CHANGE_SHARDING_ATTRIBUTESWillberaisedifthereisanattempttoupdatethevalueofashardattribute.

1470-ERROR_CLUSTER_UNSUPPORTEDWillberaisedwhenthereisanattempttocarryoutanoperationthatisnotsupportedinthecontextofashardedcollection.

1471-ERROR_CLUSTER_ONLY_ON_COORDINATORWillberaisedifthereisanattempttorunacoordinator-onlyoperationonadifferenttypeofnode.

1472-ERROR_CLUSTER_READING_PLAN_AGENCYWillberaisedifacoordinatororDBservercannotreadthePlanintheagency.

1473-ERROR_CLUSTER_COULD_NOT_TRUNCATE_COLLECTIONWillberaisedifacoordinatorcannottruncateallshardsofaclustercollection.

1474-ERROR_CLUSTER_AQL_COMMUNICATIONWillberaisediftheinternalcommunicationoftheclusterforAQLproducesanerror.

1475-ERROR_ARANGO_DOCUMENT_NOT_FOUND_OR_SHARDING_ATTRIBUTES_CHANGEDWillberaisedwhenadocumentwithagivenidentifierorhandleisunknown,oriftheshardingattributeshavebeenchangedinaREPLACEoperationinthecluster.

1476-ERROR_CLUSTER_COULD_NOT_DETERMINE_IDWillberaisedifaclusterserveratstartupcouldnotdetermineitsownIDfromthelocalinfoprovided.

1477-ERROR_CLUSTER_ONLY_ON_DBSERVERWillberaisedifthereisanattempttorunaDBserver-onlyoperationonadifferenttypeofnode.

1478-ERROR_CLUSTER_BACKEND_UNAVAILABLEWillberaisedifarequireddbservercan'tbereached.

1479-ERROR_CLUSTER_UNKNOWN_CALLBACK_ENDPOINTAnendpointcouldn'tbefound

1480-ERROR_CLUSTER_AGENCY_STRUCTURE_INVALIDThestructureintheagencyisinvalid

1481-ERROR_CLUSTER_AQL_COLLECTION_OUT_OF_SYNCWillberaisedifacollectionneededduringqueryexecutionisoutofsync.Thiscurrentlycanonlyhappenwhenusingsatellitecollections

1482-ERROR_CLUSTER_COULD_NOT_CREATE_INDEX_IN_PLANWillberaisedwhenacoordinatorinaclustercannotcreateanentryforanewindexinthePlanhierarchyintheagency.

1483-ERROR_CLUSTER_COULD_NOT_DROP_INDEX_IN_PLANWillberaisedwhenacoordinatorinaclustercannotremoveanindexfromthePlanhierarchyintheagency.

1484-ERROR_CLUSTER_CHAIN_OF_DISTRIBUTESHARDSLIKEWillberaisedifonetriestocreateacollectionwithadistributeShardsLikeattributewhichpointstoanothercollectionthatalsohasone.

1485-ERROR_CLUSTER_MUST_NOT_DROP_COLL_OTHER_DISTRIBUTESHARDSLIKEWillberaisedifonetriestodropacollectiontowhichanothercollectionpointswithitsdistributeShardsLikeattribute.

1486-ERROR_CLUSTER_UNKNOWN_DISTRIBUTESHARDSLIKEWillberaisedifonetriestocreateacollectionwhichpointstoanunknowncollectioninitsdistributeShardsLikeattribute.

1487-ERROR_CLUSTER_INSUFFICIENT_DBSERVERSWillberaisedifonetriestocreateacollectionwithareplicationFactorgreaterthantheavailablenumberofDBServers.

1488-ERROR_CLUSTER_COULD_NOT_DROP_FOLLOWERWillberaisedifafollowerthatoughttobedroppedcouldnotbedroppedintheagency(underCurrent).

1489-ERROR_CLUSTER_SHARD_LEADER_REFUSES_REPLICATIONWillberaisedifareplicationoperationisrefusedbyashardleader.

1490-ERROR_CLUSTER_SHARD_FOLLOWER_REFUSES_OPERATIONWillberaisedifanon-replicationoperationisrefusedbyashardfollower.

1491-ERROR_CLUSTER_SHARD_LEADER_RESIGNEDWillberaisedifanon-replicationoperationisrefusedbyaformershardleaderthathasfoundoutthatitisnolongertheleader.

1492-ERROR_CLUSTER_AGENCY_COMMUNICATION_FAILEDWillberaisedifaftervariousretriesanagencyoperationcouldnotbeperformedsuccessfully.

1493-ERROR_CLUSTER_DISTRIBUTE_SHARDS_LIKE_REPLICATION_FACTORWillberaisedifintendedreplicationfactordoesnotmatchthatoftheprototypeshardgiveninditributeShardsLikeparameter.

1494-ERROR_CLUSTER_DISTRIBUTE_SHARDS_LIKE_NUMBER_OF_SHARDSWillberaisedifintendednumberofshardsdoesnotmatchthatoftheprototypeshardgiveninditributeShardsLikeparameter.

1495-ERROR_CLUSTER_LEADERSHIP_CHALLENGE_ONGOINGWillberaisedwhenserversarecurrentlycompetingforleadership,andtheresultisstillunknown.

1496-ERROR_CLUSTER_NOT_LEADERWillberaisedwhenanoperationissenttoanon-leadingserver.

ArangoDBqueryerrors

1500-ERROR_QUERY_KILLEDWillberaisedwhenarunningqueryiskilledbyanexplicitadmincommand.

1501-ERROR_QUERY_PARSEWillberaisedwhenqueryisparsedandisfoundtobesyntacticallyinvalid.

1502-ERROR_QUERY_EMPTYWillberaisedwhenanemptyqueryisspecified.

1503-ERROR_QUERY_SCRIPTWillberaisedwhenaruntimeerroriscausedbythequery.

1504-ERROR_QUERY_NUMBER_OUT_OF_RANGEWillberaisedwhenanumberisoutsidetheexpectedrange.

1510-ERROR_QUERY_VARIABLE_NAME_INVALIDWillberaisedwhenaninvalidvariablenameisused.

1511-ERROR_QUERY_VARIABLE_REDECLAREDWillberaisedwhenavariablegetsre-assignedinaquery.

1512-ERROR_QUERY_VARIABLE_NAME_UNKNOWNWillberaisedwhenanunknownvariableisusedorthevariableisundefinedthecontextitisused.

1521-ERROR_QUERY_COLLECTION_LOCK_FAILEDWillberaisedwhenareadlockonthecollectioncannotbeacquired.

1522-ERROR_QUERY_TOO_MANY_COLLECTIONSWillberaisedwhenthenumberofcollectionsinaqueryisbeyondtheallowedvalue.

1530-ERROR_QUERY_DOCUMENT_ATTRIBUTE_REDECLAREDWillberaisedwhenadocumentattributeisre-assigned.

1540-ERROR_QUERY_FUNCTION_NAME_UNKNOWNWillberaisedwhenanundefinedfunctioniscalled.

1541-ERROR_QUERY_FUNCTION_ARGUMENT_NUMBER_MISMATCHWillberaisedwhenthenumberofargumentsusedinafunctioncalldoesnotmatchtheexpectednumberofargumentsforthefunction.

1542-ERROR_QUERY_FUNCTION_ARGUMENT_TYPE_MISMATCHWillberaisedwhenthetypeofanargumentusedinafunctioncalldoesnotmatchtheexpectedargumenttype.

1543-ERROR_QUERY_INVALID_REGEXWillberaisedwhenaninvalidregexargumentvalueisusedinacalltoafunctionthatexpectsaregex.

1550-ERROR_QUERY_BIND_PARAMETERS_INVALIDWillberaisedwhenthestructureofbindparameterspassedhasanunexpectedformat.

1551-ERROR_QUERY_BIND_PARAMETER_MISSINGWillberaisedwhenabindparameterwasdeclaredinthequerybutthequeryisbeingexecutedwithnovalueforthatparameter.

1552-ERROR_QUERY_BIND_PARAMETER_UNDECLAREDWillberaisedwhenavaluegetsspecifiedforanundeclaredbindparameter.

1553-ERROR_QUERY_BIND_PARAMETER_TYPEWillberaisedwhenabindparameterhasaninvalidvalueortype.

1560-ERROR_QUERY_INVALID_LOGICAL_VALUEWillberaisedwhenanon-booleanvalueisusedinalogicaloperation.

1561-ERROR_QUERY_INVALID_ARITHMETIC_VALUEWillberaisedwhenanon-numericvalueisusedinanarithmeticoperation.

1562-ERROR_QUERY_DIVISION_BY_ZEROWillberaisedwhenthereisanattempttodividebyzero.

1563-ERROR_QUERY_ARRAY_EXPECTEDWillberaisedwhenanon-arrayoperandisusedforanoperationthatexpectsanarrayargumentoperand.

1569-ERROR_QUERY_FAIL_CALLEDWillberaisedwhenthefunctionFAIL()iscalledfrominsideaquery.

1570-ERROR_QUERY_GEO_INDEX_MISSINGWillberaisedwhenageorestrictionwasspecifiedbutnosuitablegeoindexisfoundtoresolveit.

1571-ERROR_QUERY_FULLTEXT_INDEX_MISSINGWillberaisedwhenafulltextqueryisperformedonacollectionwithoutasuitablefulltextindex.

1572-ERROR_QUERY_INVALID_DATE_VALUEWillberaisedwhenavaluecannotbeconvertedtoadate.

1573-ERROR_QUERY_MULTI_MODIFYWillberaisedwhenanAQLquerycontainsmorethanonedata-modifyingoperation.

1574-ERROR_QUERY_INVALID_AGGREGATE_EXPRESSIONWillberaisedwhenanAQLquerycontainsaninvalidaggregateexpression.

1575-ERROR_QUERY_COMPILE_TIME_OPTIONSWillberaisedwhenanAQLdata-modificationquerycontainsoptionsthatcannotbefiguredoutatquerycompiletime.

1576-ERROR_QUERY_EXCEPTION_OPTIONSWillberaisedwhenanAQLdata-modificationquerycontainsaninvalidoptionsspecification.

1577-ERROR_QUERY_COLLECTION_USED_IN_EXPRESSIONWillberaisedwhenacollectionisusedasanoperandinanAQLexpression.

1578-ERROR_QUERY_DISALLOWED_DYNAMIC_CALLWillberaisedwhenadynamicfunctioncallismadetoafunctionthatcannotbecalleddynamically.

1579-ERROR_QUERY_ACCESS_AFTER_MODIFICATIONWillberaisedwhencollectiondataareaccessedafteradata-modificationoperation.

AQLuserfunctionerrors

1580-ERROR_QUERY_FUNCTION_INVALID_NAMEWillberaisedwhenauserfunctionwithaninvalidnameisregistered.

1581-ERROR_QUERY_FUNCTION_INVALID_CODEWillberaisedwhenauserfunctionisregisteredwithinvalidcode.

1582-ERROR_QUERY_FUNCTION_NOT_FOUNDWillberaisedwhenauserfunctionisaccessedbutnotfound.

1583-ERROR_QUERY_FUNCTION_RUNTIME_ERRORWillberaisedwhenauserfunctionthrowsaruntimeexception.

AQLqueryregistryerrors1590-ERROR_QUERY_BAD_JSON_PLANWillberaisedwhenanHTTPAPIforaquerygotaninvalidJSONobject.

1591-ERROR_QUERY_NOT_FOUNDWillberaisedwhenanIdofaqueryisnotfoundbytheHTTPAPI.

1592-ERROR_QUERY_IN_USEWillberaisedwhenanIdofaqueryisfoundbytheHTTPAPIbutthequeryisinuse.

ArangoDBcursorerrors

1600-ERROR_CURSOR_NOT_FOUNDWillberaisedwhenacursorisrequestedviaitsidbutacursorwiththatidcannotbefound.

1601-ERROR_CURSOR_BUSYWillberaisedwhenacursorisrequestedviaitsidbutaconcurrentrequestisstillusingthecursor.

ArangoDBtransactionerrors1650-ERROR_TRANSACTION_INTERNALWillberaisedwhenawrongusageoftransactionsisdetected.thisisaninternalerrorandindicatesabuginArangoDB.

1651-ERROR_TRANSACTION_NESTEDWillberaisedwhentransactionsarenested.

1652-ERROR_TRANSACTION_UNREGISTERED_COLLECTIONWillberaisedwhenacollectionisusedinthemiddleofatransactionbutwasnotregisteredattransactionstart.

1653-ERROR_TRANSACTION_DISALLOWED_OPERATIONWillberaisedwhenadisallowedoperationiscarriedoutinatransaction.

1654-ERROR_TRANSACTION_ABORTEDWillberaisedwhenatransactionwasaborted.

Usermanagementerrors

1700-ERROR_USER_INVALID_NAMEWillberaisedwhenaninvalidusernameisused.

1701-ERROR_USER_INVALID_PASSWORDWillberaisedwhenaninvalidpasswordisused.

1702-ERROR_USER_DUPLICATEWillberaisedwhenausernamealreadyexists.

1703-ERROR_USER_NOT_FOUNDWillberaisedwhenausernameisupdatedthatdoesnotexist.

1704-ERROR_USER_CHANGE_PASSWORDWillberaisedwhentheusermustchangehispassword.

1705-ERROR_USER_EXTERNALWillberaisedwhentheuserisauthenicatedbyanexternalserver.

Servicemanagementerrors(legacy)

ThesehavebeensupercededbytheFoxxmanagementerrorsinpublicAPIs.

1750-ERROR_SERVICE_INVALID_NAMEWillberaisedwhenaninvalidservicenameisspecified.

1751-ERROR_SERVICE_INVALID_MOUNTWillberaisedwhenaninvalidmountisspecified.

1752-ERROR_SERVICE_DOWNLOAD_FAILEDWillberaisedwhenaservicedownloadfromthecentralrepositoryfailed.

1753-ERROR_SERVICE_UPLOAD_FAILEDWillberaisedwhenaserviceuploadfromtheclienttotheArangoDBserverfailed.

LDAPerrors1800-ERROR_LDAP_CANNOT_INITcannotinitaLDAPconnection

1801-ERROR_LDAP_CANNOT_SET_OPTIONcannotsetaLDAPoption

1802-ERROR_LDAP_CANNOT_BINDcannotbindtoaLDAPserver

1803-ERROR_LDAP_CANNOT_UNBINDcannotunbindfromaLDAPserver

1804-ERROR_LDAP_CANNOT_SEARCHcannotsearchtheLDAPserver

1805-ERROR_LDAP_CANNOT_START_TLScannotstaraTLSLDAPsession

1806-ERROR_LDAP_FOUND_NO_OBJECTSLDAPdidn'tfoundanyobjectswiththespecifiedsearchquery

1807-ERROR_LDAP_NOT_ONE_USER_FOUNDLDAPfoundzerooremorethanoneuser

1808-ERROR_LDAP_USER_NOT_IDENTIFIEDLDAPfoundauser,butitsnotthedesiredone

1820-ERROR_LDAP_INVALID_MODEcantdistinguishavalidmodeforprovidedldapconfiguration

Taskerrors

1850-ERROR_TASK_INVALID_IDWillberaisedwhenataskiscreatedwithaninvalidid.

1851-ERROR_TASK_DUPLICATE_IDWillberaisedwhenataskidiscreatedwithaduplicateid.

1852-ERROR_TASK_NOT_FOUNDWillberaisedwhenataskwiththespecifiedidcouldnotbefound.

Graph/traversalerrors1901-ERROR_GRAPH_INVALID_GRAPHWillberaisedwhenaninvalidnameispassedtotheserver.

1902-ERROR_GRAPH_COULD_NOT_CREATE_GRAPHWillberaisedwhenaninvalidname,verticesoredgesispassedtotheserver.

1903-ERROR_GRAPH_INVALID_VERTEXWillberaisedwhenaninvalidvertexidispassedtotheserver.

1904-ERROR_GRAPH_COULD_NOT_CREATE_VERTEXWillberaisedwhenthevertexcouldnotbecreated.

1905-ERROR_GRAPH_COULD_NOT_CHANGE_VERTEXWillberaisedwhenthevertexcouldnotbechanged.

1906-ERROR_GRAPH_INVALID_EDGEWillberaisedwhenaninvalidedgeidispassedtotheserver.

1907-ERROR_GRAPH_COULD_NOT_CREATE_EDGEWillberaisedwhentheedgecouldnotbecreated.

1908-ERROR_GRAPH_COULD_NOT_CHANGE_EDGEWillberaisedwhentheedgecouldnotbechanged.

1909-ERROR_GRAPH_TOO_MANY_ITERATIONSWillberaisedwhentoomanyiterationsaredoneinagraphtraversal.

1910-ERROR_GRAPH_INVALID_FILTER_RESULTWillberaisedwhenaninvalidfilterresultisreturnedinagraphtraversal.

1920-ERROR_GRAPH_COLLECTION_MULTI_USEanedgecollectionmayonlybeusedonceinoneedgedefinitionofagraph.,

1921-ERROR_GRAPH_COLLECTION_USE_IN_MULTI_GRAPHSisalreadyusedbyanothergraphinadifferentedgedefinition.,

1922-ERROR_GRAPH_CREATE_MISSING_NAMEagraphnameisrequiredtocreateagraph.,

1923-ERROR_GRAPH_CREATE_MALFORMED_EDGE_DEFINITIONtheedgedefinitionismalformed.Ithastobeanarrayofobjects.,

1924-ERROR_GRAPH_NOT_FOUNDagraphwiththisnamecouldnotbefound.,

1925-ERROR_GRAPH_DUPLICATEagraphwiththisnamealreadyexists.,

1926-ERROR_GRAPH_VERTEX_COL_DOES_NOT_EXISTthespecifiedvertexcollectiondoesnotexistorisnotpartofthegraph.,

1927-ERROR_GRAPH_WRONG_COLLECTION_TYPE_VERTEXthecollectionisnotavertexcollection.,

1928-ERROR_GRAPH_NOT_IN_ORPHAN_COLLECTIONVertexcollectionnotinorphancollectionofthegraph.,

1929-ERROR_GRAPH_COLLECTION_USED_IN_EDGE_DEFThecollectionisalreadyusedinanedgedefinitionofthegraph.,

1930-ERROR_GRAPH_EDGE_COLLECTION_NOT_USEDTheedgecollectionisnotusedinanyedgedefinitionofthegraph.,

1931-ERROR_GRAPH_NOT_AN_ARANGO_COLLECTIONThecollectionisnotanArangoCollection.,

1932-ERROR_GRAPH_NO_GRAPH_COLLECTIONcollection_graphsdoesnotexist.,

1933-ERROR_GRAPH_INVALID_EXAMPLE_ARRAY_OBJECT_STRINGInvalidexampletype.HastobeString,ArrayorObject.,

1934-ERROR_GRAPH_INVALID_EXAMPLE_ARRAY_OBJECTInvalidexampletype.HastobeArrayorObject.,

1935-ERROR_GRAPH_INVALID_NUMBER_OF_ARGUMENTSInvalidnumberofarguments.Expected:,

1936-ERROR_GRAPH_INVALID_PARAMETERInvalidparametertype.,

1937-ERROR_GRAPH_INVALID_IDInvalidid,

1938-ERROR_GRAPH_COLLECTION_USED_IN_ORPHANSThecollectionisalreadyusedintheorphansofthegraph.,

1939-ERROR_GRAPH_EDGE_COL_DOES_NOT_EXISTthespecifiededgecollectiondoesnotexistorisnotpartofthegraph.,

1940-ERROR_GRAPH_EMPTYTherequestedgraphhasnoedgecollections.

Sessionerrors1950-ERROR_SESSION_UNKNOWNWillberaisedwhenaninvalid/unknownsessionidispassedtotheserver.

1951-ERROR_SESSION_EXPIREDWillberaisedwhenasessionisexpired.

SimpleClienterrors

2000-SIMPLE_CLIENT_UNKNOWN_ERRORThiserrorshouldnothappen.

2001-SIMPLE_CLIENT_COULD_NOT_CONNECTWillberaisedwhentheclientcouldnotconnecttotheserver.

2002-SIMPLE_CLIENT_COULD_NOT_WRITEWillberaisedwhentheclientcouldnotwritedata.

2003-SIMPLE_CLIENT_COULD_NOT_READWillberaisedwhentheclientcouldnotreaddata.

Communicatorerrors

2100-COMMUNICATOR_REQUEST_ABORTEDRequestwasaborted.

2101-COMMUNICATOR_DISABLEDCommunicationwasdisabled.

Foxxmanagementerrors

3000-ERROR_MALFORMED_MANIFEST_FILETheservicemanifestfileisnotwell-formedJSON.

3001-ERROR_INVALID_SERVICE_MANIFESTTheservicemanifestcontainsinvalidvalues.

3002-ERROR_SERVICE_FILES_MISSINGTheservicefolderorbundledoesnotexistonthisserver.

3003-ERROR_SERVICE_FILES_OUTDATEDThelocalservicebundledoesnotmatchthechecksuminthedatabase.

3004-ERROR_INVALID_FOXX_OPTIONSTheserviceoptionscontaininvalidvalues.

3007-ERROR_INVALID_MOUNTPOINTTheservicemountpathcontainsinvalidcharacters.

3009-ERROR_SERVICE_NOT_FOUNDNoservicefoundatthegivenmountpath.

3010-ERROR_SERVICE_NEEDS_CONFIGURATIONTheserviceismissingconfigurationordependencies.

3011-ERROR_SERVICE_MOUNTPOINT_CONFLICTAservicealreadyexistsatthegivenmountpath.

3012-ERROR_SERVICE_MANIFEST_NOT_FOUNDTheservicedirectorydoesnotcontainamanifestfile.

3013-ERROR_SERVICE_OPTIONS_MALFORMEDTheserviceoptionsarenotwell-formedJSON.

3014-ERROR_SERVICE_SOURCE_NOT_FOUNDThesourcepathdoesnotmatchafileordirectory.

3015-ERROR_SERVICE_SOURCE_ERRORThesourcepathcouldnotberesolved.

3016-ERROR_SERVICE_UNKNOWN_SCRIPTTheservicedoesnothaveascriptwiththisname.

JavaScriptmoduleloadererrors3100-ERROR_MODULE_NOT_FOUNDThemodulepathcouldnotberesolved.

3101-ERROR_MODULE_SYNTAX_ERRORThemodulecouldnotbeparsedbecauseofasyntaxerror.

3103-ERROR_MODULE_FAILUREFailedtoinvokethemoduleinitscontext.

Enterpriseerrors

4000-ERROR_NO_SMART_COLLECTIONTherequestedcollectionneedstobesmart,butitain't

4001-ERROR_NO_SMART_GRAPH_ATTRIBUTEThegivendocumentdoesnothavethesmartgraphattributeset.

4002-ERROR_CANNOT_DROP_SMART_COLLECTIONThissmartcollectioncannotbedropped,itdictatesshardinginthegraph.

4003-ERROR_KEY_MUST_BE_PREFIXED_WITH_SMART_GRAPH_ATTRIBUTEInasmartvertexcollection_keymustbeprefixedwiththevalueofthesmartgraphattribute.

4004-ERROR_ILLEGAL_SMART_GRAPH_ATTRIBUTEThegivensmartGraphattributeisillegalandconnotbeusedforsharding.Allsystemattributesareforbidden.

Agencyerrors

20001-ERROR_AGENCY_INQUIRY_SYNTAXInquiryhandlesalistofstringclientIds:[,...].

20011-ERROR_AGENCY_INFORM_MUST_BE_OBJECTTheinformmessageintheagencymustbeanobject.

20012-ERROR_AGENCY_INFORM_MUST_CONTAIN_TERMTheinformmessageintheagencymustcontainauintparameter'term'.

20013-ERROR_AGENCY_INFORM_MUST_CONTAIN_IDTheinformmessageintheagencymustcontainastringparameter'id'.

20014-ERROR_AGENCY_INFORM_MUST_CONTAIN_ACTIVETheinformmessageintheagencymustcontainanarray'active'.

20015-ERROR_AGENCY_INFORM_MUST_CONTAIN_POOLTheinformmessageintheagencymustcontainanobject'pool'.

20016-ERROR_AGENCY_INFORM_MUST_CONTAIN_MIN_PINGTheinformmessageintheagencymustcontainanobject'minping'.

20017-ERROR_AGENCY_INFORM_MUST_CONTAIN_MAX_PINGTheinformmessageintheagencymustcontainanobject'maxping'.

20018-ERROR_AGENCY_INFORM_MUST_CONTAIN_TIMEOUT_MULTTheinformmessageintheagencymustcontainanobject'timeoutMult'.

20020-ERROR_AGENCY_INQUIRE_CLIENT_ID_MUST_BE_STRINGInquirybyclientIdfailed

20021-ERROR_AGENCY_CANNOT_REBUILD_DBSWillberaisedifthereadDBorthespearHeadcannotberebuiltfromthereplicatedlog.

Supervisionerrors

20501-ERROR_SUPERVISION_GENERAL_FAILURE

Generalsupervisionfailure.

Dispatchererrors

21001-ERROR_DISPATCHER_IS_STOPPINGWillbereturnedifashutdownisinprogress.

21002-ERROR_QUEUE_UNKNOWNWillbereturnedifaqueuewiththisnamedoesnotexist.

21003-ERROR_QUEUE_FULLWillbereturnedifaqueuewiththisnameisfull.

Glossary

Collection

Acollectionconsistsofdocuments.Itisuniquelyidentifiedbyitscollectionidentifier.Italsohasauniquenamethatclientsshouldusetoidentifyandaccessit.Collectionscanberenamed.Itwillchangethecollectionname,butnotthecollectionidentifier.Collectionscontaindocumentsofaspecifictype.Therearecurrentlytwotypes:document(default)andedge.Thetypeisspecifiedbytheuserwhenthecollectioniscreated,andcannotbechangedlater.

CollectionIdentifier

Acollectionidentifieridentifiesacollectioninadatabase.Itisastringvalueandisuniquewithinthedatabase.UptoincludingArangoDB1.1,thecollectionidentifierhasbeenaclient'sprimarymeanstoaccesscollections.StartingwithArangoDB1.2,clientsshouldinsteaduseacollection'suniquenametoaccessacollectioninsteadofitsidentifier.

ArangoDBcurrentlyuses64bitunsignedintegervaluestomaintaincollectionidsinternally.Whenreturningcollectionidstoclients,ArangoDBwillputthemintoastringtoensurethecollectionidisnotclippedbyclientsthatdonotsupportbigintegers.ClientsshouldtreatthecollectionidsreturnedbyArangoDBasopaquestringswhentheystoreoruseitlocally.

CollectionName

Acollectionnameidentifiesacollectioninadatabase.Itisastringandisuniquewithinthedatabase.Unlikethecollectionidentifieritissuppliedbythecreatorofthecollection.Thecollectionnamemustconsistofletters,digits,andthe_(underscore)and-(dash)charactersonly.PleaserefertoNamingConventionsformoreinformationonvalidcollectionnames.

Database

ArangoDBcanhandlemultipledatabasesinthesameserverinstance.Databasescanbeusedtologicallygroupandseparatedata.AnArangoDBdatabaseconsistsofcollectionsanddedicateddatabase-specificworkerprocesses.

Adatabasecontainsitsowncollections(whichcannotbeaccessedfromotherdatabases),Foxxapplications,andreplicationloggersandappliers.EachArangoDBdatabasecontainsitsownsystemcollections(e.g._users,_replication,...).

TherewillalwaysbeatleastonedatabaseinArangoDB.Thisisthedefaultdatabase,named_system.Thisdatabasecannotbedropped,andprovidesspecialoperationsforcreating,dropping,andenumeratingdatabases.Userscancreateadditionaldatabasesandgivethemuniquenamestoaccessthemlater.Databasemanagementoperationscannotbeinitiatedfromoutofuser-defineddatabases.

WhenArangoDBisaccessedviaitsHTTPRESTAPI,thedatabasenameisreadfromthefirstpartoftherequestURIpath(e.g./_db/_system/...).IftherequestURIdoesnotcontainadatabasename,thedatabasenameisautomaticallyderivedfromtheendpoint.PleaserefertoDatabaseEndpointformoreinformation.

DatabaseName

AsingleArangoDBinstancecanhandlemultipledatabasesinparallel.Whenmultipledatabasesareused,eachdatabasemustbegivenauniquename.Thisnameisusedtouniquelyidentifyadatabase.ThedefaultdatabaseinArangoDBisnamed_system.

Thedatabasenameisastringconsistingofonlyletters,digitsandthe_(underscore)and-(dash)characters.User-defineddatabasenamesmustalwaysstartwithaletter.Databasenamesiscase-sensitive.

DatabaseOrganization

AsingleArangoDBinstancecanhandlemultipledatabasesinparallel.Bydefault,therewillbeatleastonedatabase,whichisnamed_system.

Databasesarephysicallystoredinseparatesub-directoriesunderneaththedatabasedirectory,whichitselfresidesintheinstance'sdatadirectory.

Glossary

Eachdatabasehasitsownsub-directory,nameddatabase-.Thedatabasedirectorycontainssub-directoriesforthecollectionsofthedatabase,andafilenamedparameter.json.Thisfilecontainsthedatabaseidandname.

InanexampleArangoDBinstancewhichhastwodatabases,thefilesystemlayoutcouldlooklikethis:

data/#theinstance'sdatadirectory

databases/#sub-directorycontainingalldatabases'data

database-<id>/#sub-directoryforasingledatabase

parameter.json#filecontainingdatabaseidandname

collection-<id>/#directorycontainingdataaboutacollection

database-<id>/#sub-directoryforanotherdatabase

parameter.json#filecontainingdatabaseidandname

collection-<id>/#directorycontainingdataaboutacollection

Foxxapplicationsarealsoorganizedindatabase-specificdirectoriesinsidetheapplicationpath.Thefilesystemlayoutcouldlooklikethis:

apps/#theinstance'sapplicationdirectory

system/#systemapplications(canbeignored)

_db/#sub-directorycontainingdatabase-specificapplications

<database-name>/#sub-directoryforasingledatabase

<mountpoint>/APP#sub-directoryforasingleapplication

<database-name>/#sub-directoryforanotherdatabase

<mountpoint>/APP#sub-directoryforasingleapplication

Document

DocumentsinArangoDBareJSONobjects.Theseobjectscanbenested(toanydepth)andmaycontainarrays.Eachdocumentisuniquelyidentifiedbyitsdocumenthandle.

DocumentEtag

Thedocumentrevision(_revvalue)enclosedindoublequotes.TherevisionisreturnedbyseveralHTTPAPImethodsintheEtagHTTPheader.

DocumentHandle

Adocumenthandleuniquelyidentifiesadocumentinthedatabase.Itisastringandconsistsofthecollection'snameandthedocumentkey(_keyattribute)separatedby/.Thedocumenthandleisstoredinadocument's_idattribute.

DocumentKey

Adocumentkeyisastringthatuniquelyidentifiesadocumentinagivencollection.Itcanandshouldbeusedbyclientswhenspecificdocumentsaresearched.Documentkeysarestoredinthe_keyattributeofdocuments.ThekeyvaluesareautomaticallyindexedbyArangoDBinacollection'sprimaryindex.Thuslookingupadocumentbyitskeyisregularlyafastoperation.The_keyvalueofadocumentisimmutableoncethedocumenthasbeencreated.

Bydefault,ArangoDBwillauto-generateadocumentkeyifno_keyattributeisspecified,andusetheuser-specified_keyvalueotherwise.

Thisbehaviorcanbechangedonaper-collectionlevelbycreatingcollectionswiththekeyOptionsattribute.

UsingkeyOptionsitispossibletodisallowuser-specifiedkeyscompletely,ortoforceaspecificregimeforauto-generatingthe_keyvalues.

Therearesomerestrictionsforuser-definedkeys(seeNamingConventionsfordocumentkeys).

DocumentRevision

Glossary

AsArangoDBsupportsMVCC,documentscanexistinmorethanonerevision.ThedocumentrevisionistheMVCCtokenusedtoidentifyaparticularrevisionofadocument.Itisastringvaluecurrentlycontaininganintegernumberandisuniquewithinthelistofdocumentrevisionsforasingledocument.Documentrevisionscanbeusedtoconditionallyupdate,replaceordeletedocumentsinthedatabase.Inordertofindaparticularrevisionofadocument,youneedthedocumenthandleandthedocumentrevision.

Thedocumentrevisionisstoredinthe_revattributeofadocument,andissetandupdatedbyArangoDBautomatically.The_revvaluecannotbesetfromtheoutside.

ArangoDBcurrentlyuses64bitunsignedintegervaluestomaintaindocumentrevisionsinternally.Whenreturningdocumentrevisionstoclients,ArangoDBwillputthemintoastringtoensuretherevisionidisnotclippedbyclientsthatdonotsupportbigintegers.ClientsshouldtreattherevisionidreturnedbyArangoDBasanopaquestringwhentheystoreoruseitlocally.ThiswillallowArangoDBtochangetheformatofrevisionidslaterifthisshouldberequired.Clientscanuserevisionsidstoperformsimpleequality/non-equalitycomparisons(e.g.tocheckwhetheradocumenthaschangedornot),buttheyshouldnotuserevisionidstoperformgreater/lessthancomparisonswiththemtocheckifadocumentrevisionisolderthanoneanother,evenifthismightworkforsomecases.

Edgesarespecialdocumentsusedforconnectingotherdocumentsintoagraph.Anedgedescribestheconnectionbetweentwodocumentsusingtheinternalattributes:_fromand_to.Thesecontaindocumenthandles,namelythestart-pointandtheend-pointoftheedge.

EdgeCollection

Edgecollectionsarecollectionsthatstoreedges.

EdgeDefinition

Edgedefinitionsarepartsofthedefinitionofnamedgraphs.Theydescribewhichedgecollectionsconnectwhichvertexcollections.

GeneralGraph

Modulemaintaininggraphsetupinthe_graphscollection-akanamedgraphs.Configureswhichedgecollectionsrelatetowhichvertexcollections.Ensuresgraphconsistencyinmodificationqueries.

NamedGraphs

Namedgraphsenforceconsistencybetweenedgecollectionsandvertexcollections,soifyouremoveavertex,edgespointingtoitwillberemovedtoo.

Indexesareusedtoallowfastaccesstodocumentsinacollection.Allcollectionshaveaprimaryindex,whichisthedocument's_keyattribute.Thisindexcannotbedroppedorchanged.

Edgecollectionswillalsohaveanautomaticallycreatededgesindex,whichcannotbemodified.Thisindexprovidesquickaccesstodocumentsviathe_fromand_toattributes.

Mostuser-landindexescanbecreatedbydefiningthenamesoftheattributeswhichshouldbeindexed.Someindextypesallowindexingjustoneattribute(e.g.fulltextindex)whereasotherindextypesallowindexingmultipleattributes.

Indexingthesystemattribute_idinuser-definedindexesisnotsupportedbyanyindextype.

EdgesIndex

Anedgesindexisautomaticallycreatedforedgecollections.Itcontainsconnectionsbetweenvertexdocumentsandisinvokedwhentheconnectingedgesofavertexarequeried.Thereisnowaytoexplicitlycreateordeleteedgesindexes.

FulltextIndex

Glossary

Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.Afulltextindexcanbedefinedononeattributeonly,andwillincludeallwordscontainedindocumentsthathaveatextualvalueintheindexattribute.SinceArangoDB2.6theindexwillalsoincludewordsfromtheindexattributeiftheindexattributeisanarrayofstrings,oranobjectwithstringvaluemembers.

Forexample,givenafulltextindexonthetranslationsattributeandthefollowingdocuments,thensearchingforлисаusingthefulltextindexwouldreturnonlythefirstdocument.SearchingfortheindexfortheexactstringFoxwouldreturnthefirsttwodocuments,andsearchingforprefix:Foxwouldreturnallthreedocuments:

{translations:{en:"fox",de:"Fuchs",fr:"renard",ru:"лиса"}}

{translations:"FoxistheEnglishtranslationoftheGermanwordFuchs"}

{translations:["ArangoDB","document","database","Foxx"]}

Iftheindexattributeisneitherastring,anobjectoranarray,itscontentswillnotbeindexed.Whenindexingthecontentsofanarrayattribute,anarraymemberwillonlybeincludedintheindexifitisastring.Whenindexingthecontentsofanobjectattribute,anobjectmembervaluewillonlybeincludedintheindexifitisastring.Otherdatatypesareignoredandnotindexed.

Onlywordswitha(specifiable)minimumlengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedbylibicu,whichistakingintoaccounttheselectedlanguageprovidedatserverstart.Wordsareindexedintheirlower-casedform.Theindexsupportscompletematchqueries(fullwords)andprefixqueries.

GeoIndex

Ageoindexisusedtofindplacesonthesurfaceoftheearthfast.

IndexHandle

Anindexhandleuniquelyidentifiesanindexinthedatabase.Itisastringandconsistsofacollectionnameandanindexidentifierseparatedby/.

HashIndex

Ahashindexisusedtofinddocumentsbasedonexamples.Ahashindexcanbecreatedforoneormultipledocumentattributes.

Ahashindexwillonlybeusedbyqueriesifallindexedattributesarepresentintheexampleorsearchquery,andifallattributesarecomparedusingtheequality(==operator).Thatmeansthehashindexdoesnotsupportrangequeries.

AuniquehashindexhasanamortizedcomplexityofO(1)forlookup,insert,update,andremoveoperations.Thenon-uniquehashindexissimilar,butamortizedlookupperformanceisO(n),withnbeingthenumberofindexentrieswiththesamelookupvalue.

SkiplistIndex

Askiplistisasortedindextypethatcanbeusedtofindrangesofdocuments.

AnonymousGraphs

Youmayuseedgecollectionswithvertexcollectionswithoutthegraphmanagementfacilities.However,graphconsistencyisnotenforcedbythese.Ifyouremovevertices,youhavetoensurebyyourselvesedgespointingtothisvertexareremoved.Anonymousgraphsmaynotbebrowsedusinggraphviewerinthewebinterface.Thismaybefasterinsomescenarios.

Aviewisconceptiallyatransformationfunctionoverdocumentsfromzeroormorecollections.Itisuniquelyidentifiedbyitsviewidentifier.Italsohasauniquenamethatclientsshouldusetoidentifyandaccessit.Viewscanberenamed.Renamingaviewwillchangetheviewname,butnottheviewidentifier.Theconceptualtransformationfunctionemployedbyaviewtypeisimplementationspecific.Thetypeisspecifiedbytheuserwhentheviewiscreated,andcannotbechangedlater.Thefollowingviewtypesarecurrentlysupported:

arangosearch

ViewIdentifier

Glossary

Aviewidentifieridentifiesaviewinadatabase.Itisastringvalueandisuniquewithinthedatabase.Clientsshoulduseaview'suniquenametoaccessaviewinsteadofitsidentifier.

ArangoDBcurrentlyuses64bitunsignedintegervaluestomaintainviewidsinternally.Whenreturningviewidstoclients,ArangoDBwillputthemintoastringtoensuretheviewidisnotclippedbyclientsthatdonotsupportbigintegers.ClientsshouldtreattheviewidsreturnedbyArangoDBasopaquestringswhentheystoreorusethemlocally.

ViewName

Aviewnameidentifiesaviewinadatabase.Itisastringandisuniquewithinthedatabase.Unliketheviewidentifieritissuppliedbythecreatoroftheview.Theviewnamemustconsistofletters,digits,andthe_(underscore)and-(dash)charactersonly.PleaserefertoNamingConventionsformoreinformationonvalidviewnames,whichfollowthesameguidelinesascollectionnames.

Glossary

download.arangodb.com€¦ · 1.1 1.2 1.2.1 1.2.1.1 1.2.1.2 1.2.1.3 1.2.1.4 1.2.2 1.2.3 1.2.4 1.3...

Documents

әдістемелік нұсқау ұсынылған › arm › upload › umk › 43222.pdf · 1.4.4.2 Екілік санау жүйесінен ауыстыру Алгаритмнің

BOLETÍN OFICIAL DO PARLAMENTO DE GALICIA · 1.4. Procedementos de información 1.4.4. Preguntas 1.4.4.2. Preguntas orais en comisión Admisión a trámite, asignación a comisión

block manendragarh ROP 2012-13cghealth.nic.in/health/programonitoring/PIPHealth... · A.1.4.4 MH- Incentive under Maternal Health 0.00 A.1.4.4.1 MH - Incentive - ASHA - JSY - Incentive

Caring for Our Children Basics Health and Safety ... · 1.4.1.1/1.4.2.3 Pre-service Training/Orientation ..... 9 1.4.3.1 First Aid and CPR Training for Staff ..... 9 1.4.4.1/1.4.4.2

· 2021. 1. 29. · SUMARIO 1. Procedementos parlamentarios 1.4. Procedementos de información 1.4.4. Preguntas 1.4.4.2. Preguntas orais en comisión Admisión a trámite, asignación

1.4.4.3 Lab - Researching IT and Networking Job Opportunities.pdf

BOLETÍN OFICIAL DO PARLAMENTO DE GALICIA · SUMARIO 1. Procedementos parlamentarios 1.4. Procedementos de información 1.4.4. Preguntas 1.4.4.3. Preguntas para resposta por escrito

Parlamento de Galiciaparlamentodegalicia.gal/sitios/web/... · 2017. 9. 4. · SUMARIO 1. Procedementos parlamentarios 1.4. Procedementos de información 1.4.4. Preguntas 1.4.4.3

BOLETÍN OFICIAL DO PARLAMENTO DE GALICIA · SUMARIO 1. Procedementos parlamentarios 1.4. Procedementos de información 1.4.4. Preguntas 1.4.4.2. Preguntas orais en Comisión ı1599

CAPÍTULO 1 INTRODUCIÓN · 1.4.4.2 O Manual de procesos e procedementos.....11 1.5. Anexos..... 15. INTRODUCIÓN AO SISTEMA MARCO DE GARANTÍA INTERNA DE CALIDADE DA UNIVERSIDADE

Xenus Regeneration Guide€¦ · 1.4.4: Regen Resistor Configuration with CME 2 1.4.4.1 Click Configure Regen ( ) to open the Regen Resistor screen. 1.4.4.2 Select a resistor option

1.4.4.3 Lab - Researching IT and Networking Job Opportunities RESUELTO

pspa.ff.unair.ac.idpspa.ff.unair.ac.id/files/downloads/akademikgasal2016-2017.pdf · Halaman FAKULTAS FARMASI UNIVERSITAS AIRLANGGA No. Dokumen : IK 1.4.4.1 RO INSTRUKSI KERJA Mulai

STATE ROP 2012-13 CHC Janakpurcghealth.nic.in/health/programonitoring/PIPHealth... · A.1.4.4 MH- Incentive under Maternal Health 0.00 A.1.4.4.1 MH - Incentive - ASHA - JSY - Incentive

Práctica de laboratorio: Investigación de oportunidades ...itroque.edu.mx/cisco/cisco1/course/files/1.4.4.3 Lab - Researching... · el cuadro del campo Job Title (Puesto) y haga

Kaufmännische Grundbildung E-Profil...Folien einfügen, löschen, kopieren, Reihenfolge wechseln Hintergrundformate bestimmen 1.4.4.1 Folien gestalten Text, Textformatierungen K3

Section [11] External Background - ClassNK · Background Document SECTION 11 – GENERAL REQUIREMENTS ... 1.4.4 Structural continuity 1.4.4.a The text of 1.4.4.1 – 1.4.4.4 is taken

Cisco 1.4.4.3

IRAUNKORTASUN DIAGNOSTIKOA ZIORTZA-BOLIBAR · 1.4.4.2 Kutsadura akustikoa..... 59 1.5 Potentzialki kontaminatuak dauden enplazamenduak62 1.5.1 EPER ERREGISTROA ETA IPPC LEGE DIREKTIBA.....62

LICENCIADO(A) EN CONTADURIA PÚBLICA - … · Tipos de Contingencias 21 1.4.4.1. Irrecuperabilidad de Cuentas por Cobrar 21 ... CAPITULO II 2 30Diseño Metodológico y Diagnóstico