Upload
others
View
11
Download
0
Embed Size (px)
Citation preview
1.1
1.2
1.2.1
1.2.1.1
1.2.1.2
1.2.1.3
1.2.1.4
1.2.2
1.2.3
1.2.4
1.3
1.4
1.4.1
1.4.2
1.4.3
1.5
1.5.1
1.5.2
1.5.2.1
1.5.2.2
1.5.3
1.5.3.1
1.5.3.2
1.5.4
1.5.4.1
1.5.4.2
1.5.4.3
1.5.5
1.5.6
1.5.6.1
1.5.6.2
1.5.6.3
1.5.6.4
1.6
1.6.1
1.6.2
1.6.3
1.6.4
1.6.4.1
1.6.4.2
1.6.4.3
TableofContentsIntroduction
GettingStarted
Installing
Linux
MacOSX
Windows
Compiling
Authentication
AccessingtheWebInterface
ComingfromSQL
Highlights
Scalability
Architecture
Datamodels
Limitations
Datamodels&modeling
Concepts
Databases
WorkingwithDatabases
NotesaboutDatabases
Collections
CollectionMethods
DatabaseMethods
Documents
BasicsandTerminology
CollectionMethods
DatabaseMethods
Graphs,Vertices&Edges
NamingConventions
DatabaseNames
CollectionNames
DocumentKeys
AttributeNames
Indexing
IndexBasics
Whichindextousewhen
IndexUtilization
WorkingwithIndexes
HashIndexes
Skiplists
Persistent
1
1.6.4.4
1.6.4.5
1.6.4.6
1.7
1.7.1
1.7.1.1
1.7.1.2
1.7.2
1.7.2.1
1.7.3
1.7.3.1
1.7.3.2
1.7.4
1.7.5
1.8
1.8.1
1.8.2
1.8.3
1.8.4
1.8.5
1.8.6
1.8.7
1.8.7.1
1.8.7.2
1.8.7.3
1.8.7.4
1.8.8
1.8.9
1.8.9.1
1.8.9.1.1
1.8.9.1.2
1.8.9.2
1.8.9.2.1
1.8.9.2.2
1.8.10
1.8.11
1.8.12
1.8.13
1.8.14
1.8.14.1
1.8.14.2
1.8.14.3
1.8.14.4
1.8.14.5
FulltextIndexes
GeoIndexes
VertexCentricIndexes
Graphs
GeneralGraphs
GraphManagement
GraphFunctions
SmartGraphs
SmartGraphManagement
Traversals
UsingTraversalObjects
ExampleData
WorkingwithEdges
Pregel
FoxxMicroservices
Ataglance
Gettingstarted
Servicemanifest
Servicecontext
Configuration
Dependencies
Routers
Endpoints
Middleware
Request
Response
UsingGraphQL
Sessionsmiddleware
Sessionstorages
Collectionstorage
JWTstorage
Sessiontransports
Cookietransport
Headertransport
Servingfiles
Writingtests
CrossOrigin
Scriptsandqueuedjobs
Migrating2.xservices
Migratingfrompre-2.8
manifest.json
applicationContext
RepositoriesandModels
Controllers
2
1.8.14.5.1
1.8.14.5.2
1.8.14.5.3
1.8.14.5.4
1.8.14.5.5
1.8.14.5.6
1.8.14.6
1.8.14.7
1.8.14.8
1.8.15
1.8.16
1.8.17
1.8.17.1
1.8.17.2
1.8.17.3
1.9
1.9.1
1.9.2
1.9.3
1.9.4
1.9.5
1.10
1.10.1
1.10.2
1.10.3
1.10.4
1.10.5
1.10.5.1
1.10.5.2
1.10.5.3
1.10.5.4
1.11
1.11.1
1.11.1.1
1.11.1.2
1.11.1.3
1.11.1.4
1.11.1.5
1.11.1.6
1.11.1.7
1.11.1.8
1.11.1.9
1.11.2
1.11.2.1
Requestcontext
Errorhandling
Before/After/Around
Requestobject
Responseobject
DependencyInjection
Sessions
AuthandOAuth2
FoxxQueries
Legacycompatibilitymode
Usermanagement
Relatedmodules
Authentication
OAuth1.0a
OAuth2.0
Transactions
Transactioninvocation
Passingparameters
Lockingandisolation
Durability
Limitations
Deployment
Singleinstance
Cluster:Mesos,DC/OS
Cluster:Generic&Docker
MultipleDatacenters
AdvancedTopics
StandaloneAgency
Cluster:Localtestsetups
Cluster:Processes
Cluster:Docker
Administration
WebInterface
Dashboard
Cluster
Collections
Document
Queries
Graphs
Services
Users
Logs
ArangoDBShell
ShellOutput
3
1.11.2.2
1.11.2.3
1.11.3
1.11.4
1.11.5
1.11.6
1.11.7
1.11.7.1
1.11.8
1.11.8.1
1.11.8.2
1.11.8.3
1.11.8.4
1.11.8.5
1.11.8.6
1.11.8.7
1.11.8.8
1.11.8.9
1.11.8.10
1.11.8.11
1.11.8.12
1.11.9
1.11.10
1.11.11
1.11.11.1
1.11.11.2
1.11.12
1.11.12.1
1.11.12.1.1
1.11.12.1.2
1.11.12.1.3
1.11.12.1.4
1.11.12.1.5
1.11.12.2
1.11.12.2.1
1.11.12.2.2
1.11.12.3
1.11.13
1.11.14
1.11.14.1
1.11.14.2
1.11.14.3
1.11.14.4
1.11.14.5
Configuration
Details
Arangoimp
Arangodump
Arangorestore
Arangoexport
ManagingUsers
InArangosh
ServerConfiguration
OperatingSystemConfiguration
ManagingEndpoints
SSLConfiguration
LDAPOptions
LoggingOptions
GeneralOptions
Write-AheadLogOptions
CompactionOptions
ClusterOptions
RocksDBEngineOptions
HashCacheOptions
AsynchronousTasks
Durability
Encryption
Auditing
Configuration
Events
Replication
AsynchronousReplication
Components
Per-DatabaseSetup
Server-LevelSetup
SyncingCollections
ReplicationLimitations
SynchronousReplication
Implementation
Configuration
SatelliteCollections
Sharding
Upgrading
Upgradingto3.3
Upgradingto3.2
Upgradingto3.1
Upgradingto3.0
Upgradingto2.8
4
1.11.14.6
1.11.14.7
1.11.14.8
1.11.14.9
1.11.14.10
1.12
1.12.1
1.12.2
1.12.3
1.12.4
1.13
1.13.1
1.13.2
1.14
1.14.1
1.14.2
1.14.3
1.14.4
1.14.5
1.14.6
1.14.7
1.14.8
1.14.9
1.14.10
1.14.11
1.14.12
1.14.13
1.14.14
1.14.15
1.14.16
1.14.17
1.14.18
1.14.19
1.14.20
1.14.21
1.14.22
1.14.23
1.15
1.15.1
1.15.1.1
1.15.1.2
1.15.2
1.15.2.1
1.15.2.2
Upgradingto2.6
Upgradingto2.5
Upgradingto2.4
Upgradingto2.3
Upgradingto2.2
Troubleshooting
arangod
EmergencyConsole
DatafileDebugger
Arangobench
Architecture
Write-aheadlog
StorageEngines
Releasenotes
WhatsNewin3.3
Incompatiblechangesin3.3
WhatsNewin3.2
KnownIssuesin3.2
Incompatiblechangesin3.2
WhatsNewin3.1
Incompatiblechangesin3.1
WhatsNewin3.0
Incompatiblechangesin3.0
WhatsNewin2.8
Incompatiblechangesin2.8
WhatsNewin2.7
Incompatiblechangesin2.7
WhatsNewin2.6
Incompatiblechangesin2.6
WhatsNewin2.5
Incompatiblechangesin2.5
WhatsNewin2.4
Incompatiblechangesin2.4
WhatsNewin2.3
Incompatiblechangesin2.3
WhatsNewin2.2
WhatsNewin2.1
Appendix
References
db
collection
JavaScriptModules
@arangodb
console
5
1.15.2.3
1.15.2.4
1.15.2.5
1.15.2.6
1.15.2.7
1.15.2.8
1.15.2.9
1.15.3
1.15.3.1
1.15.3.1.1
1.15.3.1.2
1.15.3.1.3
1.15.3.1.4
1.15.3.1.5
1.15.3.2
1.15.3.2.1
1.15.3.2.2
1.15.3.2.3
1.15.4
1.15.5
crypto
fs
request
actions
queries
Write-aheadlog
TaskManagement
Deprecated
SimpleQueries
SequentialAccess
Pagination
ModificationQueries
GeoQueries
FulltextQueries
Actions
DeliveringHTMLPages
JsonObjects
Modifying
Errorcodesandmeanings
Glossary
6
ArangoDBv3.3.2DocumentationWelcometotheArangoDBdocumentation!
NewandeagertotryoutArangoDB?Startrightawaywithourbeginner'sguide:GettingStarted
Thedocumentationisorganizedinfourhandbooks:
ThismanualdescribesArangoDBanditsfeaturesindetailforyouasauser,developerandadministrator.TheAQLhandbookexplainsArangoDB'squerylanguageAQL.TheHTTPhandbookdescribestheinternalAPIofArangoDBthatisusedtocommunicatewithclients.Ingeneral,theHTTPhandbookwillbeofinteresttodriverdevelopers.Ifyouuseanyoftheexistingdriversforthelanguageofyourchoice,youcanskipthishandbook.Ourcookbookwithrecipesforspecificproblemsandsolutions.
Featuresareillustratedwithinteractiveusageexamples;youcancut'n'pastethemintoarangoshtotrythemout.TheHTTPREST-APIfordriverdevelopersisdemonstratedwithcut'n'pasterecepiesintendedtobeusedwiththecURL.Driversmayprovidetheirownexamplesbasedonthese.jsbasedexamplestoimproveunderstandeabilityfortheirrespectiveusers,i.e.forthejavadriversomeofthesamplesarere-implemented.
Overview
ArangoDBisanativemulti-model,open-sourcedatabasewithflexibledatamodelsfordocuments,graphs,andkey-values.BuildhighperformanceapplicationsusingaconvenientSQL-likequerylanguageorJavaScriptextensions.UseACIDtransactionsifyourequirethem.Scalehorizontallyandverticallywithafewmouseclicks.
Keyfeaturesinclude:
installingArangoDBonaclusterisaseasyasinstallinganapponyourmobileFlexibledatamodeling:modelyourdataascombinationofkey-valuepairs,documentsorgraphs-perfectforsocialrelationsPowerfulquerylanguage(AQL)toretrieveandmodifydataUseArangoDBasanapplicationserverandfuseyourapplicationanddatabasetogetherformaximalthroughputTransactions:runqueriesonmultipledocumentsorcollectionswithoptionaltransactionalconsistencyandisolationReplicationandSharding:setupthedatabaseinamaster-slaveconfigurationorspreadbiggerdatasetsacrossmultipleserversConfigurabledurability:lettheapplicationdecideifitneedsmoredurabilityormoreperformanceNo-nonsensestorage:ArangoDBusesallofthepowerofmodernstoragehardware,likeSSDandlargecachesJavaScriptforall:nolanguagezoo,youcanuseonelanguagefromyourbrowsertoyourback-endArangoDBcanbeeasilydeployedasafault-tolerantdistributedstatemachine,whichcanserveastheanimalbrainofdistributedappliancesItisopensource(ApacheLicense2.0)
Community
IfyouhavequestionsregardingArangoDB,Foxx,drivers,orthisdocumentationdon'thesitatetocontactuson:
GitHubforissuesandmisbehaviororpullrequestsGoogleGroupsfordiscussionsaboutArangoDBingeneralortoannounceyournewFoxxAppStackOverflowforquestionsaboutAQL,usagescenariosetc.Slack,ourcommunitychat
Whenreportingissues,pleasedescribe:
theenvironmentyourunArangoDBintheArangoDBversionyouusewhetheryou'reusingFoxxtheclientyou'reusing
Introduction
7
whichpartsofthedocumentationyou'reworkingwith(link)whatyouexpecttohappenwhatisactuallyhappening
Wewillrespondassoonaspossible.
Introduction
8
Gettingstarted
Overview
Thisbeginner'sguidewillmakeyoufamiliarwithArangoDB.Wewillcoverhowto
installandrunalocalArangoDBserverusethewebinterfacetointeractwithitstoreexampledatainthedatabasequerythedatabasetoretrievethedataagaineditandremoveexistingdata
Installation
Headtoarangodb.com/download,selectyouroperatingsystemanddownloadArangoDB.Youmayalsofollowtheinstructionsonhowtoinstallwithapackagemanager,ifavailable.
IfyouinstalledabinarypackageunderLinux,theserverisautomaticallystarted.
IfyouinstalledArangoDBusinghomebrewunderMacOSX,starttheserverbyrunning/usr/local/sbin/arangod.
IfyouinstalledArangoDBunderWindowsasaservice,theserverisautomaticallystarted.Otherwise,runthearangod.exelocatedintheinstallationfolder'sbindirectory.YoumayhavetorunitasadministratortograntitwritepermissionstoC:\ProgramFiles.
Formorein-depthinformationonhowtoinstallArangoDB,aswellasavailablestartupparameters,installationinaclusterandsoon,seeInstalling.
ArangoDBofferstwostorageengines:MMFilesandRocksDB.Choosetheonewhichsuitsyourneedsbestintheinstallationprocessoronfirststartup.
Securingtheinstallation
Thedefaultinstallationcontainsonedatabase_systemandausernamedroot.
DebianbasedpackagesandtheWindowsinstallerwillaskforapasswordduringtheinstallationprocess.Red-Hatbasedpackageswillsetarandompassword.Forallotherinstallationpackagesyouneedtoexecute
shell>arango-secure-installation
Thiswillaskedforarootpasswordandsetsthispassword.
Webinterface
Theserveritself(arangod)speaksHTTP/REST,butyoucanusethegraphicalwebinterfacetokeepitsimple.There'salsoarangosh,asynchronousshellforinteractionwiththeserver.Ifyou'readeveloper,youmightprefertheshellovertheGUI.Itdoesnotprovidefeatureslikesyntaxhighlightinghowever.
WhenyoustartusingArangoDBinyourproject,youwilllikelyuseanofficialorcommunity-madedriverwritteninthesamelanguageasyourproject.Driversimplementaprogramminginterfacethatshouldfeelnaturalforthatprogramminglanguage,anddoallthetalkingtotheserver.Therefore,youcanmostcertainlyignoretheHTTPAPIunlessyouwanttowriteadriveryourselforexplicitlywanttousetherawinterface.
Togetfamiliarwiththedatabasesystemyoucanevenputdriversasideandusethewebinterface(codenameAardvark)forbasicinteraction.Thewebinterfacewillbecomeavailableshortlyafteryoustartedarangod.Youcanaccessitinyourbrowserathttp://localhost:8529-ifnot,pleaseseeTroubleshooting.
GettingStarted
9
Bydefault,authenticationisenabled.Thedefaultuserisroot.Dependingontheinstallationmethodused,theinstallationprocesseitherpromptedfortherootpasswordorthedefaultrootpasswordisempty(seeabove).
Nextyouwillbeaskedwhichdatabasetouse.Everyserverinstancecomeswitha_systemdatabase.Selectthisdatabasetocontinue.
GettingStarted
10
Youshouldthenbepresentedthedashboardwithserverstatisticslikethis:
Foramoredetaileddescriptionoftheinterface,seeWebInterface.
Databases,collectionsanddocuments
Databasesaresetsofcollections.Collectionsstorerecords,whicharereferredtoasdocuments.CollectionsaretheequivalentoftablesinRDBMS,anddocumentscanbethoughtofasrowsinatable.Thedifferenceisthatyoudon'tdefinewhatcolumns(orratherattributes)therewillbeinadvance.Everydocumentinanycollectioncanhavearbitraryattributekeysandvalues.Documentsinasinglecollectionwilllikelyhaveasimilarstructureinpracticehowever,butthedatabasesystemitselfdoesnotimposeitandwilloperatestableandfastnomatterhowyourdatalookslike.
Readmoreinthedata-modelconceptschapter.
Fornow,youcanstickwiththedefault_systemdatabaseandusethewebinterfacetocreatecollectionsanddocuments.StartbyclickingtheCOLLECTIONSmenuentry,thentheAddCollectiontile.Giveitaname,e.g.users,leavetheothersettingsunchanged(wewantittobeadocumentcollection)andSaveit.Anewtilelabeledusersshouldshowup,whichyoucanclicktoopen.
TherewillbeNodocumentsyet.Clickthegreencirclewiththewhiteplusontheright-handsidetocreateafirstdocumentinthiscollection.Adialogwillaskyoufora_key.YoucanleavethefieldblankandclickCreatetoletthedatabasesystemassignanautomaticallygenerated(unique)key.Notethatthe_keypropertyisimmutable,whichmeansyoucannotchangeitoncethedocumentiscreated.Whatyoucanuseasdocumentkeyisdescribedinthenamingconventions.
Anautomaticallygeneratedkeycouldbe"9883"(_keyisalwaysastring!),andthedocument_idwouldbe"users/9883"inthatcase.Asidefromafewsystemattributes,thereisnothinginthisdocumentyet.Let'saddacustomattributebyclickingtheicontotheleftof(emptyobject),thenAppend.Twoinputfieldswillbecomeavailable,FIELD(attributekey)andVALUE(attributevalue).Typenameaskeyandyournameasvalue.Appendanotherattribute,nameitageandsetittoyourage.ClickSavetopersistthechanges.IfyouclickonCollection:usersatthetopontheright-handsideoftheArangoDBlogo,thedocumentbrowserwillshowthedocumentsintheuserscollectionandyouwillseethedocumentyoujustcreatedinthelist.
Queryingthedatabase
GettingStarted
11
TimetoretrieveourdocumentusingAQL,ArangoDB'squerylanguage.Wecandirectlylookupthedocumentwecreatedviathe_id,buttherearealsootheroptions.ClicktheQUERIESmenuentrytobringupthequeryeditorandtypethefollowing(adjustthedocumentIDtomatchyourdocument):
RETURNDOCUMENT("users/9883")
ThenclickExecutetorunthequery.Theresultappearsbelowthequeryeditor:
[
{
"_key":"9883",
"_id":"users/9883",
"_rev":"9883",
"age":32,
"name":"JohnSmith"
}
]
Asyoucansee,theentiredocumentincludingthesystemattributesisreturned.DOCUMENT()isafunctiontoretrieveasingledocumentoralistofdocumentsofwhichyouknowthe_keysor_ids.Wereturntheresultofthefunctioncallasourqueryresult,whichisourdocumentinsideoftheresultarray(wecouldhavereturnedmorethanoneresultwithadifferentquery,butevenforasingledocumentasresult,westillgetanarrayatthetoplevel).
Thistypeofqueryiscalleddataaccessquery.Nodataiscreated,changedordeleted.Thereisanothertypeofquerycalleddatamodificationquery.Let'sinsertaseconddocumentusingamodificationquery:
INSERT{name:"KatieFoster",age:27}INTOusers
Thequeryisprettyself-explanatory:theINSERTkeywordtellsArangoDBthatwewanttoinsertsomething.Whattoinsert,adocumentwithtwoattributesinthiscase,followsnext.Thecurlybraces{}signifydocuments,orobjects.Whentalkingaboutrecordsinacollection,wecallthemdocuments.EncodedasJSON,wecallthemobjects.Objectscanalsobenested.Here'sanexample:
{
"name":{
"first":"Katie",
"last":"Foster"
}
}
INTOisamandatorypartofeveryINSERToperationandisfollowedbythecollectionnamethatwewanttostorethedocumentin.Notethattherearenoquotemarksaroundthecollectionname.
Ifyourunabovequery,therewillbeanemptyarrayasresultbecausewedidnotspecifywhattoreturnusingaRETURNkeyword.Itisoptionalinmodificationqueries,butmandatoryindataaccessqueries.EvenwithRETURN,thereturnvaluecanstillbeanemptyarray,e.g.ifthespecifieddocumentwasnotfound.Despitetheemptyresult,theabovequerystillcreatedanewuserdocument.Youcanverifythiswiththedocumentbrowser.
Let'saddanotheruser,butreturnthenewlycreateddocumentthistime:
INSERT{name:"JamesHendrix",age:69}INTOusers
RETURNNEW
NEWisapseudo-variable,whichreferstothedocumentcreatedbyINSERT.Theresultofthequerywilllooklikethis:
[
{
"_key":"10074",
"_id":"users/10074",
"_rev":"10074",
"age":69,
"name":"JamesHendrix"
}
]
GettingStarted
12
Nowthatwehave3usersinourcollection,howtoretrievethemallwithasinglequery?Thefollowingdoesnotwork:
RETURNDOCUMENT("users/9883")
RETURNDOCUMENT("users/9915")
RETURNDOCUMENT("users/10074")
TherecanonlybeasingleRETURNstatementhereandasyntaxerrorisraisedifyoutrytoexecuteit.TheDOCUMENT()functionoffersasecondarysignaturetospecifymultipledocumenthandles,sowecoulddo:
RETURNDOCUMENT(["users/9883","users/9915","users/10074"])
Anarraywiththe_idsofall3documentsispassedtothefunction.Arraysaredenotedbysquarebrackets[]andtheirelementsareseparatedbycommas.
Butwhatifweaddmoreusers?Wewouldhavetochangethequerytoretrievethenewlyaddedusersaswell.Allwewanttosaywithourqueryis:"Foreveryuserinthecollectionusers,returntheuserdocument".WecanformulatethiswithaFORloop:
FORuserINusers
RETURNuser
Itexpressestoiterateovereverydocumentinusersandtouseuserasvariablename,whichwecanusetorefertothecurrentuserdocument.Itcouldalsobecalleddoc,uorahuacatlguacamole,thisisuptoyou.Itisadvisabletouseashortandself-descriptivenamehowever.
Theloopbodytellsthesystemtoreturnthevalueofthevariableuser,whichisasingleuserdocument.Alluserdocumentsarereturnedthisway:
[
{
"_key":"9915",
"_id":"users/9915",
"_rev":"9915",
"age":27,
"name":"KatieFoster"
},
{
"_key":"9883",
"_id":"users/9883",
"_rev":"9883",
"age":32,
"name":"JohnSmith"
},
{
"_key":"10074",
"_id":"users/10074",
"_rev":"10074",
"age":69,
"name":"JamesHendrix"
}
]
Youmayhavenoticedthattheorderofthereturneddocumentsisnotnecessarilythesameastheywereinserted.Thereisnoorderguaranteedunlessyouexplicitlysortthem.WecanaddaSORToperationveryeasily:
FORuserINusers
SORTuser._key
RETURNuser
Thisdoesstillnotreturnthedesiredresult:James(10074)isreturnedbeforeJohn(9883)andKatie(9915).Thereasonisthatthe_keyattributeisastringinArangoDB,andnotanumber.Theindividualcharactersofthestringsarecompared.1islowerthan9andtheresultistherefore"correct".Ifwewantedtousethenumericalvalueofthe_keyattributesinstead,wecouldconvertthestringtoanumberanduseitforsorting.Therearesomeimplicationshowever.Wearebetteroffsortingsomethingelse.Howabouttheage,indescendingorder?
GettingStarted
13
FORuserINusers
SORTuser.ageDESC
RETURNuser
Theuserswillbereturnedinthefollowingorder:James(69),John(32),Katie(27).InsteadofDESCfordescendingorder,ASCcanbeusedforascendingorder.ASCisthedefaultthoughandcanbeomitted.
Wemightwanttolimittheresultsettoasubsetofusers,basedontheageattributeforexample.Let'sreturnusersolderthan30only:
FORuserINusers
FILTERuser.age>30
SORTuser.age
RETURNuser
ThiswillreturnJohnandJames(inthisorder).Katie'sageattributedoesnotfulfillthecriterion(greaterthan30),sheisonly27andthereforenotpartoftheresultset.Wecanmakeheragetoreturnheruserdocumentagain,usingamodificationquery:
UPDATE"9915"WITH{age:40}INusers
RETURNNEW
UPDATEallowstopartiallyeditanexistingdocument.ThereisalsoREPLACE,whichwouldremoveallattributes(exceptfor_keyand_id,whichremainthesame)andonlyaddthespecifiedones.UPDATEontheotherhandonlyreplacesthespecifiedattributesandkeepseverythingelseas-is.
TheUPDATEkeywordisfollowedbythedocumentkey(oradocument/objectwitha_keyattribute)toidentifywhattomodify.TheattributestoupdatearewrittenasobjectaftertheWITHkeyword.INdenotesinwhichcollectiontoperformthisoperationin,justlikeINTO(bothkeywordsareactuallyinterchangablehere).ThefulldocumentwiththechangesappliedisreturnedifweusetheNEWpseudo-variable:
[
{
"_key":"9915",
"_id":"users/9915",
"_rev":"12864",
"age":40,
"name":"KatieFoster"
}
IfweusedREPLACEinstead,thenameattributewouldbegone.WithUPDATE,theattributeiskept(thesamewouldapplytoadditionalattributesifwehadthem).
LetusrunourFILTERqueryagain,butonlyreturntheusernamesthistime:
FORuserINusers
FILTERuser.age>30
SORTuser.age
RETURNuser.name
Thiswillreturnthenamesofall3users:
[
"JohnSmith",
"KatieFoster",
"JamesHendrix"
]
Itiscalledaprojectionifonlyasubsetofattributesisreturned.Anotherkindofprojectionistochangethestructureoftheresults:
FORuserINusers
RETURN{userName:user.name,age:user.age}
GettingStarted
14
Thequerydefinestheoutputformatforeveryuserdocument.TheusernameisreturnedasuserNameinsteadofname,theagekeepstheattributekeyinthisexample:
[
{
"userName":"JamesHendrix",
"age":69
},
{
"userName":"JohnSmith",
"age":32
},
{
"userName":"KatieFoster",
"age":40
}
]
Itisalsopossibletocomputenewvalues:
FORuserINusers
RETURNCONCAT(user.name,"'sageis",user.age)
CONCAT()isafunctionthatcanjoinelementstogethertoastring.Weuseitheretoreturnastatementforeveryuser.Asyoucansee,theresultsetdoesnotalwayshavetobeanarrayofobjects:
[
"JamesHendrix'sageis69",
"JohnSmith'sageis32",
"KatieFoster'sageis40"
]
Nowlet'sdosomethingcrazy:foreverydocumentintheuserscollection,iterateoveralluserdocumentsagainandreturnuserpairs,e.g.JohnandKatie.Wecanusealoopinsidealoopforthistogetthecrossproduct(everypossiblecombinationofalluserrecords,3*3=9).Wedon'twantpairingslikeJohn+Johnhowever,solet'seliminatethemwithafiltercondition:
FORuser1INusers
FORuser2INusers
FILTERuser1!=user2
RETURN[user1.name,user2.name]
Weget6pairings.PairslikeJames+JohnandJohn+Jamesarebasicallyredundant,butfairenough:
[
["JamesHendrix","JohnSmith"],
["JamesHendrix","KatieFoster"],
["JohnSmith","JamesHendrix"],
["JohnSmith","KatieFoster"],
["KatieFoster","JamesHendrix"],
["KatieFoster","JohnSmith"]
]
Wecouldcalculatethesumofbothagesandcomputesomethingnewthisway:
FORuser1INusers
FORuser2INusers
FILTERuser1!=user2
RETURN{
pair:[user1.name,user2.name],
sumOfAges:user1.age+user2.age
}
WeintroduceanewattributesumOfAgesandaddupbothagesforthevalue:
[
GettingStarted
15
{
"pair":["JamesHendrix","JohnSmith"],
"sumOfAges":101
},
{
"pair":["JamesHendrix","KatieFoster"],
"sumOfAges":109
},
{
"pair":["JohnSmith","JamesHendrix"],
"sumOfAges":101
},
{
"pair":["JohnSmith","KatieFoster"],
"sumOfAges":72
},
{
"pair":["KatieFoster","JamesHendrix"],
"sumOfAges":109
},
{
"pair":["KatieFoster","JohnSmith"],
"sumOfAges":72
}
]
Ifwewantedtopost-filteronthenewattributetoonlyreturnpairswithasumlessthan100,weshoulddefineavariabletotemporarilystorethesum,sothatwecanuseitinaFILTERstatementaswellasintheRETURNstatement:
FORuser1INusers
FORuser2INusers
FILTERuser1!=user2
LETsumOfAges=user1.age+user2.age
FILTERsumOfAges<100
RETURN{
pair:[user1.name,user2.name],
sumOfAges:sumOfAges
}
TheLETkeywordisfollowedbythedesignatedvariablename(sumOfAges),thenthere'sa=symbolandthevalueoranexpressiontodefinewhatvaluethevariableissupposedtohave.Were-useourexpressiontocalculatethesumhere.WethenhaveanotherFILTERtoskiptheunwantedpairingsandmakeuseofthevariablewedeclaredbefore.Wereturnaprojectionwithanarrayoftheusernamesandthecalculatedage,forwhichweusethevariableagain:
[
{
"pair":["JohnSmith","KatieFoster"],
"sumOfAges":72
},
{
"pair":["KatieFoster","JohnSmith"],
"sumOfAges":72
}
]
Protip:whendefiningobjects,ifthedesiredattributekeyandthevariabletousefortheattributevaluearethesame,youcanuseashorthandnotation:{sumOfAges}insteadof{sumOfAges:sumOfAges}.
Finally,let'sdeleteoneoftheuserdocuments:
REMOVE"9883"INusers
ItdeletestheuserJohn(_key:"9883").Wecouldalsoremovedocumentsinaloop(samegoesforINSERT,UPDATEandREPLACE):
FORuserINusers
FILTERuser.age>=30
REMOVEuserINusers
GettingStarted
16
Thequerydeletesalluserswhoseageisgreaterthanorequalto30.
Howtocontinue
ThereisalotmoretodiscoverinAQLandmuchmorefunctionalitythatArangoDBoffers.Continuereadingtheotherchaptersandexperimentwithatestdatabasetofosteryourknowledge.
IfyouwanttowritemoreAQLqueriesrightnow,havealookhere:
DataQueries:dataaccessandmodificationqueriesHigh-leveloperations:detaileddescriptionsofFOR,FILTERandmoreoperationsnotshowninthisintroductionFunctions:areferenceofallprovidedfunctions
ArangoDBprogramsTheArangoDBpackagecomeswiththefollowingprograms:
arangod:TheArangoDBdatabasedaemon.ThisserverprogramisintendedtorunasadaemonprocessandtoservethevariousclientsconnectiontotheserverviaTCP/HTTP.
arangosh:TheArangoDBshell.Aclientthatimplementsaread-eval-printloop(REPL)andprovidesfunctionstoaccessandadministratetheArangoDBserver.
arangoimp:AbulkimporterfortheArangoDBserver.ItsupportsJSONandCSV.
arangodump:AtooltocreatebackupsofanArangoDBdatabaseinJSONformat.
arangorestore:AtooltoloaddataofabackupbackintoanArangoDBdatabase.
arango-dfdb:AdatafiledebuggerforArangoDB.ItisprimarilyintendedtobeusedduringdevelopmentofArangoDB.
arangobench:Abenchmarkandtesttool.Itcanbeusedforperformanceandserverfunctiontesting.
GettingStarted
17
InstallingFirstofall,downloadandinstallthecorrespondingRPMorDebianpackageorusehomebrewonMacOSX.Youcanfindpackagesforvariousoperationsystemsatourinstallsection,includinginstallersforWindows.
Howtodothatindetailisdescribedinthesubchaptersofthissection.
Onhowtosetupacluster,checkouttheDeploymentchapter.
Installing
18
LinuxVisittheofficialArangoDBinstallpageanddownloadthecorrectpackageforyourLinuxdistribution.Youcanfindbinarypackagesforthemostcommondistributionsthere.Followtheinstructionstouseyourfavoritepackagemanagerforthemajordistributions.AftersettinguptheArangoDBrepositoryyoucaneasilyinstallArangoDBusingyum,aptitude,urpmiorzypper.Debianbasedpackageswillaskforapasswordduringinstallation.ForanunattendedinstallationforDebian,seebelow.Red-Hatbasedpackageswillsetarandompasswordduringinstallation.Forotherdistributionsortochangethepassword,runarango-secure-installationtosetarootpassword.Alternatively,seeCompilingifyouwanttobuildArangoDByourself.Startupthedatabaseserver.
Normally,thisisdonebyexecutingthefollowingcommand:
unix>/etc/init.d/arangodstart
Itwillstarttheserver,anddothataswellatsystemboottime.
Tostoptheserveryoucanusethefollowingcommand:
unix>/etc/init.d/arangodstop
TheexactcommandsdependonyourLinuxdistribution.Youmayrequirerootprivilegestoexecutethesecommands.
LinuxMint
PleaseusethecorrespondingUbuntuorDebianpackages.
UnattendedInstallation
Debianbasedpackagewillaskforapasswordduringinstallation.Forunattendedinstallation,youcansetthepasswordusingthedebconfhelpers.
echoarangodb3arangodb3/passwordpasswordNEWPASSWORD|debconf-set-selections
echoarangodb3arangodb3/password_againpasswordNEWPASSWORD|debconf-set-selections
Thecommandsshouldbeexecutedpriortotheinstallation.
Red-Hatbasedpackageswillsetarandompasswordduringinstallation.Ifyouwanttoforceapassword,execute
ARANGODB_DEFAULT_ROOT_PASSWORD=NEWPASSWORDarango-secure-installation
Thecommandshouldbeexecutedaftertheinstallation.
Non-StandardInstallationIfyoucompiledArangoDBfromsourceanddidnotuseanyinstallationpackage–orusingnon-defaultlocationsand/ormultipleArangoDBinstancesonthesamehost–youmaywanttostarttheserverprocessmanually.Youcandosobyinvokingthearangodbinaryfromthecommandlineasshownbelow:
unix>/usr/local/sbin/arangod/tmp/vocbase
20ZZ-XX-YYT12:37:08Z[8145]INFOusingbuilt-inJavaScriptstartupfiles
20ZZ-XX-YYT12:37:08Z[8145]INFOArangoDB(version1.x.y)isreadyforbusiness
20ZZ-XX-YYT12:37:08Z[8145]INFOHaveFun!
Linux
19
Tostopthedatabaseservergracefully,youcaneitherpressCTRL-CorbysendtheSIGINTsignaltotheserverprocess.Onmanysystemsthiscanbeachievedwiththefollowingcommand:
unix>kill-2`pidofarangod`
Onceyoustartedtheserver,thereshouldbearunninginstanceofarangod-theArangoDBdatabaseserver.
unix>psauxw|fgreparangod
arangodb145360.10.6530726423464s002S1:21pm0:00.18/usr/local/sbin/arangod
Ifthereisnosuchprocess,checkthelogfile/var/log/arangodb/arangod.logforerrors.Ifyouseealogmessagelike
2012-12-03T11:35:29Z[12882]ERRORDatabasedirectoryversion(1)islowerthanserverversion(1.2).
2012-12-03T11:35:29Z[12882]ERRORItseemslikeyouhaveupgradedtheArangoDBbinary.Ifthisiswhatyouwantedtodo,pleas
erestartwiththe--database.auto-upgradeoptiontoupgradethedatainthedatabasedirectory.
2012-12-03T11:35:29Z[12882]FATALDatabaseversioncheckfailed.Pleasestarttheserverwiththe--database.auto-upgradeopti
on
makesuretostarttheserveroncewiththe--database.auto-upgradeoption.
Notethatyoumayhavetoenableloggingfirst.Ifyoustarttheserverinashell,youshouldseeerrorsloggedthereaswell.
Linux
20
MacOSXThepreferredmethodforinstallingArangoDBunderMacOSXishomebrew.However,incaseyouarenotusinghomebrew,weprovideacommand-lineapporgraphicalappwhichcontainsalltheexecutables.
Homebrew
Ifyouareusinghomebrew,thenyoucaninstallthelatestreleasedstableversionofArangoDBusingbrewasfollows:
brewinstallarangodb
ThiswillinstallthecurrentstableversionofArangoDBandalldependencieswithinyourHomebrewtree.Notethattheserverwillbeinstalledas:
/usr/local/sbin/arangod
Youcanstarttheserverbyrunningthecommand/usr/local/sbin/arangod&.
Configurationfileislocatedat
/usr/local/etc/arangodb3/arangod.conf
TheArangoDBshellwillbeinstalledas:
/usr/local/bin/arangosh
YoucanuninstallArangoDBusing:
brewuninstallarangodb
However,incaseyoustartedArangoDBusingthelaunchctl,youneedtounloaditbeforeuninstallingtheserver:
launchctlunload~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist
ThenremovetheLaunchAgent:
rm~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist
Note:IfthelatestArangoDBVersionisnotshowninhomebrew,youalsoneedtoupdatehomebrew:
brewupdate
Knownissues
Performance-theLLVMdeliveredasofMacOSXElCapitanbuildsslowbinaries.UseGCCinstead,untilthisissuehasbeenfixedbyApple.theCommandlineargumentparsingdoesn'tacceptblanksinfilenames;theCLIversionbelowdoes.ifyouneedtochangeserverendpointwhilestartinghomebrewversion,youcaneditarangod.conffileanduncommentlinewithendpointneeded,e.g.:
[server]
endpoint=tcp://0.0.0.0:8529
MacOSX
21
GraphicalApp
Incaseyouarenotusinghomebrew,wealsoprovideagraphicalapp.Youcandownloaditfromhere.
ChooseMacOSX.DownloadandinstalltheapplicationArangoDBinyourapplicationfolder.
CommandlineAppIncaseyouarenotusinghomebrew,wealsoprovideacommand-lineapp.Youcandownloaditfromhere.
ChooseMacOSX.DownloadandinstalltheapplicationArangoDB-CLIinyourapplicationfolder.
Startingtheapplicationwillstarttheserverandopenaterminalwindowshowingyouthelog-file.
ArangoDBserverhasbeenstarted
Thedatabasedirectoryislocatedat
'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/lib/arangodb'
Thelogfileislocatedat
'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/log/arangodb/arangod.log'
Youcanaccesstheserverusingabrowserat'http://127.0.0.1:8529/'
orstarttheArangoDBshell
'/Applications/ArangoDB-CLI.app/Contents/MacOS/arangosh'
Switchingtolog-filenow,killingthiswindowswillNOTstoptheserver.
2013-10-27T19:42:04Z[23840]INFOArangoDB(version1.4.devel[darwin])isreadyforbusiness.Havefun!
Notethatitispossibletoinstallboth,thehomebrewversionandthecommand-lineapp.Youshould,however,edittheconfigurationfilesofoneversionandchangetheportused.
MacOSX
22
WindowsThedefaultinstallationdirectoryisC:\ProgramFiles\ArangoDB-3.x.x.Duringtheinstallationprocessyoumaychangethis.InthefollowingdescriptionwewillassumethatArangoDBhasbeeninstalledinthelocation<ROOTDIR>.
Youhavetobecarefulwhenchoosinganinstallationdirectory.Youneedeitherwritepermissiontothisdirectoryoryouneedtomodifytheconfigurationfilefortheserverprocess.InthelattercasethedatabasedirectoryandtheFoxxdirectoryhavetobewritablebytheuser.
Single-andMultiuserInstallation
TherearetwomainmodesfortheinstallerofArangoDB.Theinstallerletsyouselect:
multiuserinstallation(default;adminprivilegesrequired)WillinstallArangoDBasservice.singleuserinstallationAllowtoinstallArangodbasnormaluser.Requiresmanualstartingofthedatabaseserver.
CheckBoxesThecheckboxesallowyoutochoseweatheryouwantto:
chosecustominstallpathsdoanautomaticupgradekeepanbackupofyourdataaddexecutablestopathcreateadesktopicon
ornot.
CustomInstallPaths
Thischeckboxcontrolsifyouwillbeabletooverridethedefaultpathsfortheinstallationinsubsequentsteps.
Thedefaultinstallationpathsare:
MultiUserDefault:
Installation:C:\ProgramFiles\ArangoDB-3.x.xDataBase:C:\ProgramData\ArangoDBFoxxService:C:\ProgramData\ArangoDB-apps
SingleUserDefault:
Installation:C:\Users\\\AppData\Local\ArangoDB-3.x.xDataBase:C:\Users\\\AppData\Local\ArangoDBFoxxService:C:\Users\\\AppData\Local\ArangoDB-apps
Wearenotusingtheroamingpartoftheuser'sprofile,becausedoingsoavoidsthedatabeingsyncedtothewindowsdomaincontroller.
AutomaticUpgrade
Ifthischeckboxisselectedtheinstallerwillattempttoperformanautomaticupdate.FormoreinformationpleaseseeUpgradingfromPreviousVersion.
KeepBackup
Selectthistocreateabackupofyourdatabasedirectoryduringautomaticupgrade.Thebackupwillbecreatednexttoyourcurrentdatabasedirectorysuffixedbyatimestamp.
Windows
23
AddtoPath
Selectthistoaddthebinarydirectorytoyoursystem'spath(multiuserinstallation)oruser'spath(singleuserinstallation).
DesktopIcon
SelectifyouwanttheinstallertocreateDesktopIconsthatletyou:
accessthewebintefacestartthecommandlineclient(arangosh)startthedatabaseserver(singleuserinstallationonly)
UpgradingfromPreviousVersionIfyouareupgradingArangoDBfromanearlierversionyouneedtocopyyourolddatabasedirectorytothenewdefaultpaths.Upgradingwillkeepyourolddata,passwordandchoiceofstorageengineasitis.SwitchingtotheRocksDBstorageenginerequiresaexportandreimportofyourdata.
StartingIfyouinstalledArangoDBformultipleusers(asaservice)itisautomaticallystarted.OtherwiseyouneedtousethelinkthatwascreatedonyouDesktopifyouchosetolettheinstallercreatedesktopiconsor
theexecutablearangod.exelocatedin<ROOTDIR>\bin.Thiswillusetheconfigurationfilearangod.conflocatedin<ROOTDIR>\etc\arangodb,whichyoucanadjusttoyourneedsandusethedatadirectory<ROOTDIR>\var\lib\arangodb.Thisistheplacewhereallyourdata(databasesandcollections)willbestoredbydefault.
Pleasechecktheoutputofthearangod.exeexecutablebeforegoingon.Iftheserverstartedsuccessfully,youshouldseealineArangoDBisreadyforbusiness.Havefun!attheendofitsoutput.
Wenowwishtocheckthattheinstallationisworkingcorrectlyandtodothiswewillbeusingtheadministrationwebinterface.Executearangod.exeifyouhavenotalreadydoneso,thenopenupyourwebbrowserandpointittothepage:
http://127.0.0.1:8529/
AdvancedStarting
Ifyouwanttoprovideourownstartscripts,youcansettheenvironmentvariableARANGODB_CONFIG_PATH.Thisvariableshouldpointtoadirectorycontainingtheconfigurationfiles.
UsingtheClient
ToconnecttoanalreadyrunningArangoDBserverinstance,thereisashellarangosh.exelocatedin<ROOTDIR>\bin.Thisstartsashellwhichcanbeused–amongstotherthings–toadministerandqueryalocalorremoteArangoDBserver.
Notethatarangosh.exedoesNOTstartaseparateserver,itonlystartstheshell.Touseityoumusthaveaserverrunningsomewhere,e.g.byusingthearangod.exeexecutable.
arangosh.exeusesconfigurationfromthefilearangosh.conflocatedin<ROOTDIR>\etc\arangodb\.Pleaseadjustthistoyourneedsifyouwanttousedifferentconnectionsettingsetc.
Uninstalling
TouninstalltheArangoserverapplicationyoucanusethewindowscontrolpanel(asyouwouldnormallyuninstallanapplication).Notehowever,thatanydatafilescreatedbytheArangoserverwillremainaswellasthe<ROOTDIR>directory.Tocompletetheuninstallationprocess,removethedatafilesandthe<ROOTDIR>directorymanually.
Windows
24
LimitationsforCygwin
PleasenotesomeimportantlimitationswhenrunningArangoDBunderCygwin:StartingArangoDBcanbestartedfromoutofaCygwinterminal,butpressingCTRL-Cwillforcefullykilltheserverprocesswithoutgivingitachancetohandlethekillsignal.Inthiscase,aregularservershutdownisnotpossible,whichmayleaveafileLOCKaroundintheserver'sdatadirectory.ThisfileneedstoberemovedmanuallytomakeArangoDBstartagain.Additionally,asArangoDBdoesnothaveachancetohandlethekillsignal,theservercannotforcefullyflushanydatatodiskonshutdown,leadingtopotentialdataloss.WhenstartingArangoDBfromaCygwinterminalitmightalsohappenthatnoerrorsareprintedintheterminaloutput.StartingArangoDBfromanMS-DOScommandpromptdoesnotimposetheselimitationsandisthusthepreferredmethod.
PleasenotethatArangoDBusesUTF-8asitsinternalencodingandthatthesystemconsolemustsupportaUTF-8codepage(65001)andfont.ItmaybenecessarytomanuallyswitchtheconsolefonttoafontthatsupportsUTF-8.
Windows
25
CompilingArangoDBfromscratchThefollowingsectionsdescribehowtocompileandbuildtheArangoDBfromscratch.ArangoDBwillcompileonmostLinuxandMacOSXsystems.WeassumethatyouusetheGNUC/C++compilerorclang/clang++tocompilethesource.ArangoDBhasbeentestedwiththesecompilers,butshouldbeabletocompilewithanyPosix-compliant,C++11-enabledcompiler.PleaseletusknowwhetheryousuccessfullycompileditwithanotherC/C++compiler.
Bydefault,cloningthegithubrepositorywillcheckoutdevel.ThisversioncontainsthedevelopmentversionoftheArangoDB.UsethisbranchifyouwanttomakechangestotheArangoDBsource.
OnWindowsyoufirstneedtoallowandenablesymlinksforyouruser.
PleasecheckoutthecookbookonhowtocompileArangoDB.
Compiling
26
AuthenticationArangoDBallowstorestrictaccesstodatabasestocertainusers.Allusersofthesystemdatabaseareconsideredadministrators.Duringinstallationadefaultuserrootiscreated,whichhasaccesstoalldatabases.
Youshouldcreateadatabaseforyourapplicationtogetherwithauserthathasaccessrightstothisdatabase.SeeManagingUsers.
Usethearangoshtocreateanewdatabaseanduser.
arangosh>db._createDatabase("example");
arangosh>varusers=require("@arangodb/users");
arangosh>users.save("root@example","password");
arangosh>users.grantDatabase("root@example","example");
Youcannowconnecttothenewdatabaseusingtheuserroot@example.
shell>arangosh--server.username"root@example"--server.databaseexample
Authentication
27
AccessingtheWebInterfaceArangoDBcomeswithabuilt-inwebinterfaceforadministration.ThewebinterfacecanbeaccessedviatheURL:
http://127.0.0.1:8529
Ifeverythingworksasexpected,youshouldseetheloginview:
FormoreinformationontheArangoDBwebinterface,seeWebInterface
AccessingtheWebInterface
28
ComingfromSQLIfyouworkedwitharelationaldatabasemanagementsystem(RDBMS)suchasMySQL,MariaDBorPostgreSQL,youwillbefamiliarwithitsquerylanguage,adialectofSQL(StructuredQueryLanguage).
ArangoDB'squerylanguageiscalledAQL.Therearesomesimilaritiesbetweenbothlanguagesdespitethedifferentdatamodelsofthedatabasesystems.ThemostnotabledifferenceisprobablytheconceptofloopsinAQL,whichmakesitfeelmorelikeaprogramminglanguage.Itsuitestheschema-lessmodelmorenaturalandmakesthequerylanguageverypowerfulwhileremainingeasytoreadandwrite.
TogetstartedwithAQL,havealookatourdetailedcomparisonofSQLandAQL.ItwillalsohelpyoutotranslateSQLqueriestoAQLwhenmigratingtoArangoDB.
Howdobrowsevectorstranslateintodocumentqueries?
IntraditionalSQLyoumayeitherfetchallcolumnsofatablerowbyrow,usingSELECT*FROMtable,orselectasubsetofthecolumns.Thelistoftablecolumnstofetchiscommonlycalledcolumnlistorbrowsevector:
SELECTcolumnA,columnB,columnZFROMtable
Sincedocumentsaren'ttwo-dimensional,andneitherdoyouwanttobelimitedtoreturningtwo-dimensionallists,therequirementsforaquerylanguagearehigher.AQListhusalittlebitmorecomplexthanplainSQLatfirst,butoffersmuchmoreflexibilityinthelongrun.Itletsyouhandlearbitrarilystructureddocumentsinconvenientways,mostlyleanedonthesyntaxusedinJavaScript.
Composingthedocumentstobereturned
TheAQLRETURNstatementreturnsoneitemperdocumentitishanded.Youcanreturnthewholedocument,orjustpartsofit.GiventhatoneDocumentisadocument(retrievedlikeLEToneDocument=DOCUMENT("myusers/3456789")forinstance),itcanbereturnedas-islikethis:
RETURNoneDocument
[
{
"_id":"myusers/3456789",
"_key":"3456789"
"_rev":"14253647",
"firstName":"John",
"lastName":"Doe",
"address":{
"city":"Gotham",
"street":"RoadToNowhere1"
},
"hobbies":[
{name:"swimming",howFavorite:10},
{name:"biking",howFavorite:6},
{name:"programming",howFavorite:4}
]
}
]
Returnthehobbiessub-structureonly:
RETURNoneDocument.hobbies
[
[
{name:"swimming",howFavorite:10},
{name:"biking",howFavorite:6},
{name:"programming",howFavorite:4}
ComingfromSQL
29
]
]
Returnthehobbiesandtheaddress:
RETURN{
hobbies:oneDocument.hobbies,
address:oneDocument.address
}
[
{
hobbies:[
{name:"swimming",howFavorite:10},
{name:"biking",howFavorite:6},
{name:"programming",howFavorite:4}
],
address:{
"city":"Gotham",
"street":"RoadToNowhere1"
}
}
]
Returnthefirsthobbyonly:
RETURNoneDocument.hobbies[0].name
[
"swimming"
]
Returnalistofallhobbystrings:
RETURN{hobbies:oneDocument.hobbies[*].name}
[
{hobbies:["swimming","biking","porgramming"]}
]
MorecomplexarrayandobjectmanipulationscanbedoneusingAQLfunctionsandoperators.
ComingfromSQL
30
Highlights
Version3.3
EnterpriseEdition
DatacentertoDatacenterReplication:ReplicatetheentirestructureandcontentofanArangoDBclusterasynchronouslytoanotherclusterinadifferentdatacenterwithArangoSync.Multi-datacentersupportmeansyoucanfallbacktoareplicaofyourclusterincaseofadisasterinonedatacenter.
EncryptedBackups:ArangodumpcancreatebackupsencryptedwithasecretkeyusingAES256blockcipher.
AllEditions
Server-levelReplication:Inadditiontoper-databasereplication,thereisnowanadditionalglobalApplier.Starttheglobalreplicationontheslaveonceandallcurrentandfuturedatabaseswillbereplicatedfromthemastertotheslaveautomatically.
AsynchronousFailover:Makeasingleserverinstanceresilientwithasecondserverinstance,oneasmasterandtheotherasasynchronouslyreplicatingslave,withautomaticfailovertotheslaveifthemastergoesdown.
AlsoseeWhat'sNewin3.3.
Version3.2
RocksDBStorageEngine:YoucannowuseasmuchdatainArangoDBasyoucanfitonyourdisk.Plus,youcanenjoyperformanceboostsonwritesbyhavingonlydocument-levellocks
Pregel:WeimplementeddistributedgraphprocessingwithPregeltodiscoverhiddenpatterns,identifycommunitiesandperformin-depthanalyticsoflargegraphdatasets.
Fault-TolerantFoxx:TheFoxxmanagementinternalshavebeenrewrittenfromthegrounduptomakesuremulti-coordinatorclustersetupsalwayskeeptheirservicesinsyncandnewcoordinatorsarefullyinitializedevenwhenallexistingcoordinatorsareunavailable.
Enterprise:Workingwithsomeofourlargestcustomers,we’veaddedfurthersecurityandscalabilityfeaturestoArangoDBEnterpriselikeLDAPintegration,EncryptionatRest,andthebrandnewSatelliteCollections.
AlsoseeWhat'sNewin3.2.
Version3.1SmartGraphs:Scalewithgraphstoaclusterandstayperformant.WithSmartGraphsyoucanusethe"smartness"ofyourapplicationlayertoshardyourgraphefficientlytoyourmachinesandlettraversalsrunlocally.
EncryptionControl:ChooseyourlevelofSSLencryption
Auditing:KeepadetailedlogofalltheimportantthingsthathappenedinArangoDB.
AlsoseeWhat'sNewin3.1.
Version3.0
self-organizingclusterwithsynchronousreplication,master/mastersetup,sharednothingarchitecture,clustermanagementagency.
Deeplyintegrated,nativeAQLgraphtraversal
Highlights
31
VelocyPackasnewinternalbinarystorageformataswellasforintermediateAQLvalues.
PersistentindexesviaRocksDBsuitableforsortingandrangequeries.
Foxx3.0:overhauledJSframeworkfordata-centricmicroservices
SignificantlyimprovedWebInterface
AlsoseeWhat'sNewin3.0.
Highlights
32
ScalabilityArangoDBisadistributeddatabasesupportingmultipledatamodels,andcanthusbescaledhorizontally,thatis,byusingmanyservers,typicallybasedoncommodityhardware.Thisapproachnotonlydeliversperformanceaswellascapacityimprovements,butalsoachievesresiliencebymeansofreplicationandautomaticfail-over.Furthermore,onecanbuildsystemsthatscaletheircapacitydynamicallyupanddownautomaticallyaccordingtodemand.
OnecanalsoscaleArangoDBvertically,thatis,byusingeverlargerservers.ThereisnobuiltinlimitationinArangoDB,forexample,theserverwillautomaticallyusemorethreadsifmoreCPUsarepresent.
However,scalingverticallyhasthedisadvantagethatthecostsgrowfasterthanlinearwiththesizeoftheserver,andnoneoftheresilienceanddynamicalcapabilitiescanbeachievedinthisway.
InthischapterweexplainthedistributedarchitectureofArangoDBanddiscussitsscalabilityfeaturesandlimitations:
ArangoDB'sdistributedarchitectureDifferentdatamodelsandscalabilityLimitations
Scalability
33
Architecture
TheclusterarchitectureofArangoDBisaCPmaster/mastermodelwithnosinglepointoffailure.With"CP"wemeanthatinthepresenceofanetworkpartition,thedatabaseprefersinternalconsistencyoveravailability.With"master/master"wemeanthatclientscansendtheirrequeststoanarbitrarynode,andexperiencethesameviewonthedatabaseregardless."Nosinglepointoffailure"meansthattheclustercancontinuetoserverequests,evenifonemachinefailscompletely.
Inthisway,ArangoDBhasbeendesignedasadistributedmulti-modeldatabase.Thissectiongivesashortoutlineontheclusterarchitectureandhowtheabovefeaturesandcapabilitiesareachieved.
StructureofanArangoDBcluster
AnArangoDBclusterconsistsofanumberofArangoDBinstanceswhichtalktoeachotheroverthenetwork.Theyplaydifferentroles,whichwillbeexplainedindetailbelow.Thecurrentconfigurationoftheclusterisheldinthe"Agency",whichisahighly-availableresilientkey/valuestorebasedonanoddnumberofArangoDBinstancesrunningRaftConsensusProtocol.
ForthevariousinstancesinanArangoDBclusterthereare4distinctroles:Agents,Coordinators,PrimaryandSecondaryDBservers.Inthefollowingsectionswewillshedlightoneachofthem.NotethatthetasksforallrolesrunthesamebinaryfromthesameDockerimage.
Agents
OneormultipleAgentsformtheAgencyinanArangoDBcluster.TheAgencyisthecentralplacetostoretheconfigurationinacluster.Itperformsleaderelectionsandprovidesothersynchronizationservicesforthewholecluster.WithouttheAgencynoneoftheothercomponentscanoperate.
Whilegenerallyinvisibletotheoutsideitistheheartofthecluster.Assuch,faulttoleranceisofcourseamusthavefortheAgency.ToachievethattheAgentsareusingtheRaftConsensusAlgorithm.ThealgorithmformallyguaranteesconflictfreeconfigurationmanagementwithintheArangoDBcluster.
AtitscoretheAgencymanagesabigconfigurationtree.Itsupportstransactionalreadandwriteoperationsonthistree,andotherserverscansubscribetoHTTPcallbacksforallchangestothetree.
Coordinators
Coordinatorsshouldbeaccessiblefromtheoutside.Thesearetheonestheclientstalkto.TheywillcoordinateclustertaskslikeexecutingqueriesandrunningFoxxservices.Theyknowwherethedataisstoredandwilloptimizewheretorunusersuppliedqueriesorpartsthereof.Coordinatorsarestatelessandcanthuseasilybeshutdownandrestartedasneeded.
PrimaryDBservers
PrimaryDBserversaretheoneswherethedataisactuallyhosted.Theyhostshardsofdataandusingsynchronousreplicationaprimarymayeitherbeleaderorfollowerforashard.
Theyshouldnotbeaccessedfromtheoutsidebutindirectlythroughthecoordinators.Theymayalsoexecutequeriesinpartorasawholewhenaskedbyacoordinator.
Secondaries
SecondaryDBserversareasynchronousreplicasofprimaries.Ifoneisusingonlysynchronousreplication,onedoesnotneedsecondariesatall.Foreachprimary,therecanbeoneormoresecondaries.Sincethereplicationworksasynchronously(eventualconsistency),thereplicationdoesnotimpedetheperformanceoftheprimaries.Ontheotherhand,theirreplicaofthedatacanbeslightlyoutofdate.Thesecondariesareperfectlysuitableforbackupsastheydon'tinterferewiththenormalclusteroperation.
ClusterID
Everynon-AgencyArangoDBinstanceinaclusterisassignedauniqueIDduringitsstartup.UsingitsIDanodeisidentifiablethroughoutthecluster.AllclusteroperationswillcommunicateviathisID.
Architecture
34
Sharding
UsingtherolesoutlinedaboveanArangoDBclusterisabletodistributedatainsocalledshardsacrossmultipleprimaries.Fromtheoutsidethisprocessisfullytransparentandassuchweachievethegoalsofwhatothersystemscall"master-masterreplication".InanArangoDBclusteryoutalktoanycoordinatorandwheneveryoureadorwritedataitwillautomaticallyfigureoutwherethedataisstored(read)ortobestored(write).TheinformationabouttheshardsissharedacrossthecoordinatorsusingtheAgency.
AlsoseeShardingintheAdministrationchapter.
Manysensibleconfigurations
Thisarchitectureisveryflexibleandthusallowsmanyconfigurations,whicharesuitablefordifferentusagescenarios:
1. ThedefaultconfigurationistorunexactlyonecoordinatorandoneprimaryDBserveroneachmachine.Thisachievestheclassicalmaster/mastersetup,sincethereisaperfectsymmetrybetweenthedifferentnodes,clientscanequallywelltalktoanyoneofthecoordinatorsandallexposethesameviewtothedatastore.
2. OnecandeploymorecoordinatorsthanDBservers.ThisisasensibleapproachifoneneedsalotofCPUpowerfortheFoxxservices,becausetheyrunonthecoordinators.
3. OnecandeploymoreDBserversthancoordinatorsifmoredatacapacityisneededandthequeryperformanceisthelesserbottleneck
4. Onecandeployacoordinatoroneachmachinewhereanapplicationserver(e.g.anode.jsserver)runs,andtheAgentsandDBserversonaseparatesetofmachineselsewhere.Thisavoidsanetworkhopbetweentheapplicationserverandthedatabaseandthusdecreaseslatency.Essentially,thismovessomeofthedatabasedistributionlogictothemachinewheretheclientruns.
Theseforshallsufficefornow.TheimportantpieceofinformationhereisthatthecoordinatorlayercanbescaledanddeployedindependentlyfromtheDBserverlayer.
Replication
ArangoDBofferstwowaysofdatareplicationwithinacluster,synchronousandasynchronous.Inthissectionweexplainsomedetailsandhighlighttheadvantagesanddisadvantagesrespectively.
Synchronousreplicationwithautomaticfail-over
Synchronousreplicationworksonaper-shardbasis.Oneconfiguresforeachcollection,howmanycopiesofeachshardarekeptinthecluster.Atanygiventime,oneofthecopiesisdeclaredtobethe"leader"andallotherreplicasare"followers".WriteoperationsforthisshardarealwayssenttotheDBserverwhichhappenstoholdtheleadercopy,whichinturnreplicatesthechangestoallfollowersbeforetheoperationisconsideredtobedoneandreportedbacktothecoordinator.Readoperationsareallservedbytheserverholdingtheleadercopy,thisallowstoprovidesnapshotsemanticsforcomplextransactions.
IfaDBserverfailsthatholdsafollowercopyofashard,thentheleadercannolongersynchronizeitschangestothatfollower.Afterashorttimeout(3seconds),theleadergivesuponthefollower,declaresittobeoutofsync,andcontinuesservicewithoutthefollower.Whentheserverwiththefollowercopycomesback,itautomaticallyresynchronizesitsdatawiththeleaderandsynchronousreplicationisrestored.
IfaDBserverfailsthatholdsaleadercopyofashard,thentheleadercannolongerserveanyrequests.ItwillnolongersendaheartbeattotheAgency.Therefore,asupervisionprocessrunningintheRaftleaderoftheAgency,cantakethenecessaryaction(after15secondsofmissingheartbeats),namelytopromoteoneoftheserversthatholdin-syncreplicasoftheshardtoleaderforthatshard.ThisinvolvesareconfigurationintheAgencyandleadstothefactthatcoordinatorsnowcontactadifferentDBserverforrequeststothisshard.Serviceresumes.Theothersurvivingreplicasautomaticallyresynchronizetheirdatawiththenewleader.WhentheDBserverwiththeoriginalleadercopycomesback,itnoticesthatitnowholdsafollowerreplica,resynchronizesitsdatawiththenewleaderandorderisrestored.
Allsharddatasynchronizationsaredoneinanincrementalway,suchthatresynchronizationsarequick.Thistechnologyallowstomoveshards(followerandleaderones)betweenDBserverswithoutserviceinterruptions.Therefore,anArangoDBclustercanmoveallthedataonaspecificDBservertootherDBserversandthenshutdownthatserverinacontrolledway.ThisallowstoscaledownanArangoDBclusterwithoutserviceinterruption,lossoffaulttoleranceordataloss.Furthermore,onecanre-balancethedistributionoftheshards,eithermanuallyorautomatically.
AlltheseoperationscanbetriggeredviaaREST/JSONAPIorviathegraphicalwebUI.Allfail-overoperationsarecompletelyhandledwithintheArangoDBcluster.
Architecture
35
Obviously,synchronousreplicationinvolvesacertainincreasedlatencyforwriteoperations,simplybecausethereisonemorenetworkhopwithintheclusterforeveryrequest.Thereforetheusercansetthereplicationfactorto1,whichmeansthatonlyonecopyofeachshardiskept,therebyswitchingoffsynchronousreplication.Thisisasuitablesettingforlessimportantoreasilyrecoverabledataforwhichlowlatencywriteoperationsmatter.
Asynchronousreplicationwithautomaticfail-over
Asynchronousreplicationworksdifferently,inthatitisorganizedusingprimaryandsecondaryDBservers.Eachsecondaryserverreplicatesallthedataheldonaprimarybypollinginanasynchronousway.Thisprocesshasverylittleimpactontheperformanceoftheprimary.Thedisadvantageisthatthereisadelaybetweentheconfirmationofawriteoperationthatissenttotheclientandtheactualreplicationofthedata.Ifthemasterserverfailsduringthisdelay,thencommittedandconfirmeddatacanbelost.
Nevertheless,wealsoofferautomaticfail-overwiththissetup.Contrarytothesynchronouscase,herethefail-overmanagementisdonefromoutsidetheArangoDBcluster.InafutureversionwemightmovethismanagementintothesupervisionprocessintheAgency,butasofnow,themanagementisdoneviatheMesosframeworkschedulerforArangoDB(seebelow).
ThegranularityofthereplicationisawholeArangoDBinstancewithalldatathatresidesonthatinstance,whichmeansthatyouneedtwiceasmanyinstancesaswithoutasynchronousreplication.Synchronousreplicationismoreflexibleinthatrespect,youcanhavesmallerandlargerinstances,andifonefails,thedatacanberebalancedacrosstheremainingones.
Microservicesandzeroadministation
ThedesignandcapabilitiesofArangoDBaregearedtowardsusageinmodernmicroservicearchitecturesofapplications.WiththeFoxxservicesitisveryeasytodeployadatacentricmicroservicewithinanArangoDBcluster.
Inaddition,onecandeploymultipleinstancesofArangoDBwithinthesameproject.Onepartoftheprojectmightneedascalabledocumentstore,anothermightneedagraphdatabase,andyetanothermightneedthefullpowerofamulti-modeldatabaseactuallymixingthevariousdatamodels.Thereareenormousefficiencybenefitstobereapedbybeingabletouseasingletechnologyforvariousrolesinaproject.
TosimplifyliveofthedevopsinsuchascenariowetryasmuchaspossibletouseazeroadministrationapproachforArangoDB.ArunningArangoDBclusterisresilientagainstfailuresandessentiallyrepairsitselfincaseoftemporaryfailures.Seethenextsectionforfurthercapabilitiesinthisdirection.
ApacheMesosintegration
Forthedistributedsetup,weusetheApacheMesosinfrastructurebydefault.ArangoDBisafullycertifiedpackageforDC/OSandcanthusbedeployedessentiallywithafewmouseclicksorasinglecommand,onceyouhaveanexistingDC/OScluster.ButevenonaplainApacheMesosclusteronecandeployArangoDBviaMarathonwithasingleAPIcallandsomeJSONconfiguration.
Theadvantageofthisapproachisthatwecannotonlyimplementtheinitialdeployment,butalsothelatermanagementofautomaticreplacementoffailedinstancesandthescalingoftheArangoDBcluster(triggeredmanuallyorevenautomatically).SinceallmanipulationsareeitherviathegraphicalwebUIorviaJSON/RESTcalls,onecanevenimplementauto-scalingveryeasily.
ADC/OSclusterisaverynaturalenvironmenttodeploymicroservicearchitectures,sinceitissoconvenienttodeployvariousservices,includingpotentiallymultipleArangoDBclusterinstanceswithinthesameDC/OScluster.Thebuilt-inservicediscoverymakesitextremelysimpletoconnectthevariousmicroservicesandMesosautomaticallytakescareofthedistributionanddeploymentofthevarioustasks.
SeetheDeploymentchapteranditssubsectionsforinstructions.
ItispossibletodeployanArangoDBclusterbysimplylaunchingabunchofDockercontainerswiththerightcommandlineoptionstolinkthemup,orevenonasinglemachinestartingmultipleArangoDBprocesses.Inthatcase,synchronousreplicationwillworkwithinthedeployedArangoDBcluster,andautomaticfail-overinthesensethatthedutiesofafailedserverwillautomaticallybeassignedtoanother,survivingone.However,sincetheArangoDBclustercannotwithinitselflaunchadditionalinstances,replacementoffailednodesisnotautomaticandscalingupanddownhastobemanagedmanually.Thisiswhywedonotrecommendthissetupforproductiondeployment.
Architecture
36
Differentdatamodelsandscalability
InthissectionwediscussscalabilityinthecontextofthedifferentdatamodelssupportedbyArangoDB.
Key/valuepairs
Thekey/valuestoredatamodelistheeasiesttoscale.InArangoDB,thisisimplementedinthesensethatadocumentcollectionalwayshasaprimarykey_keyattributeandintheabsenceoffurthersecondaryindexesthedocumentcollectionbehaveslikeasimplekey/valuestore.
Theonlyoperationsthatarepossibleinthiscontextaresinglekeylookupsandkey/valuepairinsertionsandupdates.If_keyistheonlyshardingattributethentheshardingisdonewithrespecttotheprimarykeyandalltheseoperationsscalelinearly.Iftheshardingisdoneusingdifferentshardkeys,thenalookupofasinglekeyinvolvesaskingallshardsandthusdoesnotscalelinearly.
Documentstore
Forthedocumentstorecaseeveninthepresenceofsecondaryindexesessentiallythesameargumentsapply,sinceanindexforashardedcollectionissimplythesameasalocalindexforeachshard.Therefore,singledocumentoperationsstillscalelinearlywiththesizeofthecluster,unlessaspecialshardingconfigurationmakeslookupsorwriteoperationsmoreexpensive.
ForadeeperanalysisofthistopicseethisblogpostinwhichgoodlinearscalabilityofArangoDBforsingledocumentoperationsisdemonstrated.
Complexqueriesandjoins
TheAQLquerylanguageallowscomplexqueries,usingmultiplecollections,secondaryindexesaswellasjoins.Inparticularwiththelatter,scalingcanbeachallenge,sinceifthedatatobejoinedresidesondifferentmachines,alotofcommunicationhastohappen.TheAQLqueryexecutionengineorganizesadatapipelineacrosstheclustertoputtogethertheresultsinthemostefficientway.Thequeryoptimizerisawareoftheclusterstructureandknowswhatdataiswhereandhowitisindexed.Therefore,itcanarriveataninformeddecisionaboutwhatpartsofthequeryoughttorunwhereinthecluster.
Nevertheless,forcertaincomplicatedjoins,therearelimitsastowhatcanbeachieved.
Graphdatabase
Graphdatabasesareparticularlygoodatqueriesongraphsthatinvolvepathsinthegraphofanaprioriunknownlength.Forexample,findingtheshortestpathbetweentwoverticesinagraph,orfindingallpathsthatmatchacertainpatternstartingatagivenvertexaresuchexamples.
However,iftheverticesandedgesalongtheoccurringpathsaredistributedacrossthecluster,thenalotofcommunicationisnecessarybetweennodes,andperformancesuffers.Toachievegoodperformanceatscale,itisthereforenecessarytogetthedistributionofthegraphdataacrosstheshardsintheclusterright.Mostofthetime,theapplicationdevelopersandusersofArangoDBknowbest,howtheirgraphsarastructured.Therefore,ArangoDBallowsuserstospecify,accordingtowhichattributesthegraphdataissharded.Ausefulfirststepisusuallytomakesurethattheedgesoriginatingatavertexresideonthesameclusternodeasthevertex.
Datamodels
37
Limitations
ArangoDBhasnobuilt-inlimitationstohorizontalscalability.ThecentralresilientAgencywilleasilysustainhundredsofDBserversandcoordinators,andtheusualdatabaseoperationsworkcompletelydecentrallyanddonotrequireassistanceoftheAgency.
Likewise,thesupervisionprocessintheAgencycaneasilydealwithlotsofservers,sinceallitsactivitiesarenotperformancecritical.
Obviously,anArangoDBclusterislimitedbytheavailableresourcesofCPU,memory,diskandnetworkbandwidthandlatency.
Limitations
38
Datamodels&modelingThischapterintroducesArangoDB'scoreconceptsandcovers
itsdatamodel(ordatamodelsrespectively),theterminologyusedthroughoutthedatabasesystemandinthisdocumentation,aswellasaspectstoconsiderwhenmodelingyourdatatostrikeabalancebetweennaturaldatastructuresandgreatperformance
Youwillalsofindusageexamplesonhowtointeractwiththedatabasesystemusingarangosh,e.g.howtocreateanddropdatabases/collections,orhowtosave,update,replaceandremovedocuments.Youcandoallthisusingthewebinterfaceaswellandmaythereforeskipthesesectionsasbeginner.
Datamodels&modeling
39
Concepts
DatabaseInteraction
ArangoDBisadatabasethatservesdocumentstoclients.ThesedocumentsaretransportedusingJSONviaaTCPconnection,usingtheHTTPprotocol.ARESTAPIisprovidedtointeractwiththedatabasesystem.
ThewebinterfacethatcomeswithArangoDB,calledAardvark,providesgraphicaluserinterfacethatiseasytouse.Aninteractiveshell,calledArangosh,isalsoshipped.Inaddition,therearesocalleddriversthatmakeiteasytousethedatabasesysteminvariousenvironmentsandprogramminglanguages.AllthesetoolsusetheHTTPinterfaceoftheserverandremovethenecessitytorollownlow-levelcodeforbasiccommunicationinmostcases.
DatamodelThedocumentsyoucanstoreinArangoDBcloselyfollowtheJSONformat,althoughtheyarestoredinabinaryformatcalledVelocyPack.Adocumentcontainszeroormoreattributes,eachoftheseattributeshavingavalue.Avaluecaneitherbeanatomictype,i.e.number,string,booleanornull,oracompoundtype,i.e.anarrayorembeddeddocument/object.Arraysandsub-objectscancontainallofthesetypes,whichmeansthatarbitrarilynesteddatastructurescanberepresentedinasingledocument.
Documentsaregroupedintocollections.Acollectioncontainszeroormoredocuments.Ifyouarefamiliarwithrelationaldatabasemanagementsystems(RDBMS)thenitissafetocomparecollectionstotablesanddocumentstorows.ThedifferenceisthatinatraditionalRDBMS,youhavetodefinecolumnsbeforeyoucanstorerecordsinatable.Suchdefinitionsarealsoknownasschemas.ArangoDBisschema-less,whichmeansthatthereisnoneedtodefinewhatattributesadocumentcanhave.Everysingledocumentcanhaveacompletelydifferentstructureandstillbestoredtogetherwithotherdocumentsinasinglecollection.Inpractice,therewillbecommondenominatorsamongthedocumentsinacollection,butthedatabasesystemitselfdoesn'tforceyoutolimityourselftoacertaindatastructure.
Therearetwotypesofcollections:documentcollection(alsoreferedtoasvertexcollectionsinthecontextofgraphs)aswellasedgecollections.Edgecollectionsstoredocumentsaswell,buttheyincludetwospecialattributes,_fromand_to,whichareusedtocreaterelationsbetweendocuments.Usually,twodocuments(vertices)storedindocumentcollectionsarelinkedbyadocument(edge)storedinanedgecollection.ThisisArangoDB'sgraphdatamodel.Itfollowsthemathematicalconceptofadirected,labeledgraph,exceptthatedgesdon'tjusthavelabels,butarefull-blowndocuments.
Collectionsexistinsideofdatabases.Therecanbeoneormanydatabases.Differentdatabasesareusuallyusedformultitenantsetups,asthedatainsidethem(collections,documentsetc.)isisolatedfromoneanother.Thedefaultdatabase_systemisspecial,becauseitcannotberemoved.Databaseusersaremanagedinthisdatabase,andtheircredentialsarevalidforalldatabasesofaserverinstance.
DataRetrievalQueriesareusedtofilterdocumentsbasedoncertaincriteria,tocomputenewdata,aswellastomanipulateordeleteexistingdocuments.Queriescanbeassimpleasa"querybyexample"orascomplexas"joins"usingmanycollectionsortraversinggraphstructures.TheyarewrittenintheArangoDBQueryLanguage(AQL).
Cursorsareusedtoiterateovertheresultofqueries,sothatyougeteasilyprocessablebatchesinsteadofonebighunk.
Indexesareusedtospeedupsearches.Therearevarioustypesofindexes,suchashashindexesandgeoindexes.
Concepts
40
HandlingDatabasesThisisanintroductiontomanagingdatabasesinArangoDBfromwithinJavaScript.
WhenyouhaveanestablishedconnectiontoArangoDB,thecurrentdatabasecanbechangedexplicitlyusingthedb._useDatabase()method.Thiswillswitchtothespecifieddatabase(provideditexistsandtheusercanconnecttoit).Fromthispointon,anyfollowingactioninthesameshellorconnectionwillusethespecifieddatabase,unlessotherwisespecified.
Note:Ifthedatabaseischanged,clientdriversneedtostorethecurrentdatabasenameontheirside,too.ThisisbecauseconnectionsinArangoDBdonotcontainanystateinformation.AllstateinformationiscontainedintheHTTPrequest/responsedata.
Toconnecttoaspecificdatabaseafterarangoshhasstartedusethecommanddescribedabove.Itisalsopossibletospecifyadatabasenamewheninvokingarangosh.Forthispurpose,usethecommand-lineparameter--server.database,e.g.
>arangosh--server.databasetest
Pleasenotethatcommands,actions,scriptsorAQLqueriesshouldneveraccessmultipledatabases,eveniftheyexist.TheonlyintendedandsupportedwayinArangoDBistouseonedatabaseatatimeforacommand,anaction,ascriptoraquery.Operationsstartedinonedatabasemustnotswitchthedatabaselaterandcontinueoperatinginanother.
Databases
41
WorkingwithDatabases
DatabaseMethods
ThefollowingmethodsareavailabletomanagedatabasesviaJavaScript.Pleasenotethatseveralofthesemethodscanbeusedfromthe_systemdatabaseonly.
Name
returnthedatabasenamedb._name()
Returnsthenameofthecurrentdatabaseasastring.
Examples
arangosh>require("@arangodb").db._name();
_system
ID
returnthedatabaseiddb._id()
Returnstheidofthecurrentdatabaseasastring.
Examples
arangosh>require("@arangodb").db._id();
1
Path
returnthepathtodatabasefilesdb._path()
Returnsthefilesystempathofthecurrentdatabaseasastring.
Examples
arangosh>require("@arangodb").db._path();
/tmp/arangosh_o42qDv/tmp-8367-1657904327/data/databases/database-1
isSystem
returnthedatabasetypedb._isSystem()
Returnswhetherthecurrentlyuseddatabaseisthe_systemdatabase.Thesystemdatabasehassomespecialprivilegesandproperties,forexample,databasemanagementoperationssuchascreateordropcanonlybeexecutedfromwithinthisdatabase.Additionally,the_systemdatabaseitselfcannotbedropped.
UseDatabase
changethecurrentdatabasedb._useDatabase(name)
Changesthecurrentdatabasetothedatabasespecifiedbyname.Notethatthedatabasespecifiedbynamemustalreadyexist.
Changingthedatabasemightbedisallowedinsomecontexts,forexampleserver-sideactions(includingFoxx).
WorkingwithDatabases
42
Whenperformingthiscommandfromarangosh,thecurrentcredentials(usernameandpassword)willbere-used.Thesecredentialsmightnotbevalidtoconnecttothedatabasespecifiedbyname.Additionally,thedatabaseonlybeaccessedfromcertainendpointsonly.Inthiscase,switchingthedatabasemightnotwork,andtheconnection/sessionshouldbeclosedandrestartedwithdifferentusernameandpasswordcredentialsand/orendpointdata.
ListDatabases
returnthelistofallexistingdatabasesdb._databases()
Returnsthelistofalldatabases.Thismethodcanonlybeusedfromwithinthe_systemdatabase.
CreateDatabase
createanewdatabasedb._createDatabase(name,options,users)
Createsanewdatabasewiththenamespecifiedbyname.Therearerestrictionsfordatabasenames(seeDatabaseNames).
Notethatevenifthedatabaseiscreatedsuccessfully,therewillbenochangeintothecurrentdatabasetothenewdatabase.Changingthecurrentdatabasemustexplicitlyberequestedbyusingthedb._useDatabasemethod.
Theoptionsattributecurrentlyhasnomeaningandisreservedforfutureuse.
Theoptionalusersattributecanbeusedtocreateinitialusersforthenewdatabase.Ifspecified,itmustbealistofuserobjects.Eachuserobjectcancontainthefollowingattributes:
username:theusernameasastring.Thisattributeismandatory.passwd:theuserpasswordasastring.Ifnotspecified,thenitdefaultstoanemptystring.active:abooleanflagindicatingwhethertheuseraccountshouldbeactiveornot.Thedefaultvalueistrue.extra:anoptionalJSONobjectwithextrauserinformation.ThedatacontainedinextrawillbestoredfortheuserbutnotbeinterpretedfurtherbyArangoDB.
Ifnoinitialusersarespecified,adefaultuserrootwillbecreatedwithanemptystringpassword.ThisensuresthatthenewdatabasewillbeaccessibleviaHTTPafteritiscreated.
Youcancreateusersinadatabaseifnoinitialuserisspecified.Switchintothenewdatabase(usernameandpasswordmustbeidenticaltothecurrentsession)andaddormodifyuserswiththefollowingcommands.
require("@arangodb/users").save(username,password,true);
require("@arangodb/users").update(username,password,true);
require("@arangodb/users").remove(username);
Alternatively,youcanspecifyuserdatadirectly.Forexample:
db._createDatabase("newDB",{},[{username:"newUser",passwd:"123456",active:true}])
Thosemethodscanonlybeusedfromwithinthe_systemdatabase.
DropDatabase
dropanexistingdatabasedb._dropDatabase(name)
Dropsthedatabasespecifiedbyname.Thedatabasespecifiedbynamemustexist.
Note:Droppingdatabasesisonlypossiblefromwithinthe_systemdatabase.The_systemdatabaseitselfcannotbedropped.
Databasesaredroppedasynchronously,andwillbephysicallyremovedifallclientshavedisconnectedandreferenceshavebeengarbage-collected.
Enginestatistics
retrievestatisticsrelatedtothestorageengine-rocksdbdb._engineStats()
Returnssomestatisticsrelatedtostorageengineactivity,includingfiguresaboutdatasize,cacheusage,etc.
WorkingwithDatabases
43
Note:CurrentlythisonlyproducesusefuloutputfortheRocksDBengine.
WorkingwithDatabases
44
NotesaboutDatabasesPleasekeepinmindthateachdatabasecontainsitsownsystemcollections,whichneedtobesetupwhenadatabaseiscreated.Thiswillmakethecreationofadatabasetakeawhile.
Replicationisconfiguredonaper-databaselevel,meaningthatanyreplicationloggingorapplyingforanewdatabasemustbeconfiguredexplicitlyafteranewdatabasehasbeencreated.
Foxxapplicationsarealsoavailableonlyinthecontextofthedatabasetheyhavebeeninstalledin.AnewdatabasewillonlyprovideaccesstothesystemapplicationsshippedwithArangoDB(thatisthewebinterfaceatthemoment)andnootherFoxxapplicationsuntiltheyareexplicitlyinstalledfortheparticulardatabase.
NotesaboutDatabases
45
JavaScriptInterfacetoCollectionsThisisanintroductiontoArangoDB'sinterfaceforcollectionsandhowtohandlecollectionsfromtheJavaScriptshellarangosh.ForotherlanguagesseethecorrespondinglanguageAPI.
Themostimportantcallisthecalltocreateanewcollection.
AddressofaCollection
AllcollectionsinArangoDBhaveauniqueidentifierandauniquename.ArangoDBinternallyusesthecollection'suniqueidentifiertolookupcollections.Thisidentifier,however,ismanagedbyArangoDBandtheuserhasnocontroloverit.Inordertoallowuserstousetheirownnames,eachcollectionalsohasauniquenamewhichisspecifiedbytheuser.Toaccessacollectionfromtheuserperspective,thecollectionnameshouldbeused,i.e.:
Collection
db._collection(collection-name)
Acollectioniscreatedbya"db._create"call.
Forexample:Assumethatthecollectionidentifieris7254820andthenameisdemo,thenthecollectioncanbeaccessedas:
db._collection("demo")
Ifnocollectionwithsuchanameexists,thennullisreturned.
Thereisashort-cutthatcanbeusedfornon-systemcollections:
Collectionname
db.collection-name
Thiscallwilleitherreturnthecollectionnameddb.collection-nameorcreateanewonewiththatnameandasetofdefaultproperties.
Note:Creatingacollectionontheflyusingdb.collection-nameisnotrecommendanddoesnotworkinarangosh.Tocreateanewcollection,pleaseuse
Create
db._create(collection-name)
Thiscallwillcreateanewcollectioncalledcollection-name.ThismethodisadatabasemethodandisdocumentedindetailatDatabaseMethods
Synchronousreplication
StartinginArangoDB3.0,thedistributedversionofferssynchronousreplication,whichmeansthatthereistheoptiontoreplicatealldataautomaticallywithintheArangoDBcluster.Thisisconfiguredforshardedcollectionsonapercollectionbasisbyspecifyinga"replicationfactor"whenthecollectioniscreated.Areplicationfactorofkmeansthataltogetherkcopiesofeachshardarekeptintheclusteronkdifferentservers,andarekeptinsync.Thatis,everywriteoperationisautomaticallyreplicatedonallcopies.
Thisisorganisedusingaleader/followermodel.Atalltimes,oneoftheserversholdingreplicasforashardis"theleader"andallothersare"followers",thisconfigurationisheldintheAgency(seeScalabilityfordetailsoftheArangoDBclusterarchitecture).Everywriteoperationissenttotheleaderbyoneofthecoordinators,andthenreplicatedtoallfollowersbeforetheoperationisreportedtohavesucceeded.Theleaderkeepsarecordofwhichfollowersarecurrentlyinsync.Incaseofnetworkproblemsorafailureofafollower,aleadercanandwilldropafollowertemporarilyafter3seconds,suchthatservicecanresume.Induecourse,thefollowerwillautomaticallyresynchronizewiththeleadertorestoreresilience.
Collections
46
Ifaleaderfails,theclusterAgencyautomaticallyinitiatesafailoverroutineafteraround15seconds,promotingoneofthefollowerstoleader.Theotherfollowers(andtheformerleader,whenitcomesback),automaticallyresynchronizewiththenewleadertorestoreresilience.Usually,thiswholefailoverprocedurecanbehandledtransparentlyforthecoordinator,suchthattheusercodedoesnotevenseeanerrormessage.
Obviously,thisfaulttolerancecomesatacostofincreasedlatency.Eachwriteoperationneedsanadditionalnetworkroundtripforthesynchronousreplicationofthefollowers,butallreplicationoperationstoallfollowershappenconcurrently.Thisis,whythedefaultreplicationfactoris1,whichmeansnoreplication.
Fordetailsonhowtoswitchonsynchronousreplicationforacollection,seethedatabasemethoddb._create(collection-name)inthesectionaboutDatabaseMethods.
Collections
47
CollectionMethods
Drop
dropsacollectioncollection.drop(options)
Dropsacollectionandallitsindexesanddata.Inordertodropasystemcollection,anoptionsobjectwithattributeisSystemsettotruemustbespecified.
Note:droppingacollectioninacluster,whichisprototypeforsharinginothercollectionsisprohibited.Inordertobeabletodropsuchacollection,alldependentcollectionsmustbedroppedfirst.
Examples
arangosh>col=db.example;
[ArangoCollection16807,"example"(typedocument,statusloaded)]
arangosh>col.drop();
arangosh>col;
[ArangoCollection16807,"example"(typedocument,statusdeleted)]
arangosh>col=db._example;
[ArangoCollection16818,"_example"(typedocument,statusloaded)]
arangosh>col.drop({isSystem:true});
arangosh>col;
[ArangoCollection16818,"_example"(typedocument,statusdeleted)]
Truncate
truncatesacollectioncollection.truncate()
Truncatesacollection,removingalldocumentsbutkeepingallitsindexes.
Examples
Truncatesacollection:
arangosh>col=db.example;
arangosh>col.save({"Hello":"World"});
arangosh>col.count();
arangosh>col.truncate();
arangosh>col.count();
showexecutionresults
Properties
getsorsetsthepropertiesofacollectioncollection.properties()Returnsanobjectcontainingallcollectionproperties.
waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheMMFilesstorageengineonly.isVolatile:IftruethenthecollectiondatawillbekeptinmemoryonlyandArangoDBwillnotwriteorsyncthedatatodisk.ThisoptionismeaningfulfortheMMFilesstorageengineonly.keyOptions(optional)additionaloptionsforkeygeneration.ThisisaJSONarraycontainingthefollowingattributes(note:someoftheattributesareoptional):
type:thetypeofthekeygeneratorusedforthecollection.allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,
CollectionMethods
48
thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeofdocumentsisconsideredanerror.increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
indexBuckets:numberofbucketsintowhichindexesusingahashtablearesplit.Thedefaultis16andthisnumberhastobeapowerof2andlessthanorequalto1024.ThisoptionismeaningfulfortheMMFilesstorageengineonly.Forverylargecollectionsoneshouldincreasethistoavoidlongpauseswhenthehashtablehastobeinitiallybuiltorresized,sincebucketsareresizedindividuallyandcanbeinitiallybuiltinparallel.Forexample,64mightbeasensiblevalueforacollectionwith100000000documents.Currently,onlytheedgeindexrespectsthisvalue,butotherindextypesmightfollowinfutureArangoDBversions.Changes(seebelow)areappliedwhenthecollectionisloadedthenexttime.Inaclustersetup,theresultwillalsocontainthefollowingattributes:numberOfShards:thenumberofshardsofthecollection.shardKeys:containsthenamesofdocumentattributesthatareusedtodeterminethetargetshardfordocuments.replicationFactor:determineshowmanycopiesofeachshardarekeptondifferentDBServers.collection.properties(properties)Changesthecollectionproperties.propertiesmustbeanobjectwithoneormoreofthefollowingattribute(s):waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheMMFilesstorageengineonly.indexBuckets:Seeabove,changesareonlyappliedwhenthecollectionisloadedthenexttime.ThisoptionismeaningfulfortheMMFilesstorageengineonly.replicationFactor:ChangethenumberofshardcopieskeptondifferentDBServers,validvaluesareintegernumbersintherangeof1-10(Clusteronly)Note:itisnotpossibletochangethejournalsizeafterthejournalordatafilehasbeencreated.Changingthisparameterwillonlyeffectnewlycreatedjournals.Alsonotethatyoucannotlowerthejournalsizetolessthensizeofthelargestdocumentalreadystoredinthecollection.Note:someothercollectionproperties,suchastype,isVolatile,orkeyOptionscannotbechangedoncethecollectioniscreated.
Examples
Readallproperties
arangosh>db.example.properties();
showexecutionresultsChangeaproperty
arangosh>db.example.properties({waitForSync:true});
showexecutionresults
Figures
returnsthefiguresofacollectioncollection.figures()
Returnsanobjectcontainingstatisticsaboutthecollection.Note:Retrievingthefigureswillalwaysloadthecollectionintomemory.
alive.count:Thenumberofcurrentlyactivedocumentsinalldatafilesandjournalsofthecollection.Documentsthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.alive.size:Thetotalsizeinbytesusedbyallactivedocumentsofthecollection.Documentsthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.dead.count:Thenumberofdeaddocuments.Thisincludesdocumentversionsthathavebeendeletedorreplacedbyanewerversion.Documentsdeletedorreplacedthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.dead.size:Thetotalsizeinbytesusedbyalldeaddocuments.dead.deletion:Thetotalnumberofdeletionmarkers.Deletionmarkersonlycontainedinthewrite-aheadlogarenotreportinginthisfigure.datafiles.count:Thenumberofdatafiles.datafiles.fileSize:Thetotalfilesizeofdatafiles(inbytes).journals.count:Thenumberofjournalfiles.journals.fileSize:Thetotalfilesizeofthejournalfiles(inbytes).
CollectionMethods
49
compactors.count:Thenumberofcompactorfiles.compactors.fileSize:Thetotalfilesizeofthecompactorfiles(inbytes).shapefiles.count:Thenumberofshapefiles.Thisvalueisdeprecatedandkeptforcompatibilityreasonsonly.Thevaluewillalwaysbe0sinceArangoDB2.0andhigher.shapefiles.fileSize:Thetotalfilesizeoftheshapefiles.Thisvalueisdeprecatedandkeptforcompatibilityreasonsonly.Thevaluewillalwaysbe0inArangoDB2.0andhigher.shapes.count:Thetotalnumberofshapesusedinthecollection.Thisincludesshapesthatarenotinuseanymore.Shapesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.shapes.size:Thetotalsizeofallshapes(inbytes).Thisincludesshapesthatarenotinuseanymore.Shapesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.attributes.count:Thetotalnumberofattributesusedinthecollection.Note:thevalueincludesdataofattributesthatarenotinuseanymore.Attributesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.attributes.size:Thetotalsizeoftheattributedata(inbytes).Note:thevalueincludesdataofattributesthatarenotinuseanymore.Attributesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.indexes.count:Thetotalnumberofindexesdefinedforthecollection,includingthepre-definedindexes(e.g.primaryindex).indexes.size:Thetotalmemoryallocatedforindexesinbytes.lastTick:Thetickofthelastmarkerthatwasstoredinajournalofthecollection.Thismightbe0ifthecollectiondoesnotyethaveajournal.uncollectedLogfileEntries:Thenumberofmarkersinthewrite-aheadlogforthiscollectionthathavenotbeentransferredtojournalsordatafiles.documentReferences:ThenumberofreferencestodocumentsindatafilesthatJavaScriptcodecurrentlyholds.Thisinformationcanbeusedfordebuggingcompactionandunloadissues.waitingFor:Anoptionalstringvaluethatcontainsinformationaboutwhichobjecttypeisattheheadofthecollection'scleanupqueue.Thisinformationcanbeusedfordebuggingcompactionandunloadissues.compactionStatus.time:Thepointintimethecompactionforthecollectionwaslastexecuted.Thisinformationcanbeusedfordebuggingcompactionissues.compactionStatus.message:Theactionthatwasperformedwhenthecompactionwaslastrunforthecollection.Thisinformationcanbeusedfordebuggingcompactionissues.
Note:collectiondatathatarestoredinthewrite-aheadlogonlyarenotreportedintheresults.Whenthewrite-aheadlogiscollected,documentsmightbeaddedtojournalsanddatafilesofthecollection,whichmaymodifythefiguresofthecollection.AlsonotethatwaitingForandcompactionStatusmaybeemptywhencalledonacoordinatorinacluster.
Additionally,thefilesizesofcollectionandindexparameterJSONfilesarenotreported.Thesefilesshouldnormallyhaveasizeofafewbyteseach.PleasealsonotethatthefileSizevaluesarereportedinbytesandreflectthelogicalfilesizes.Somefilesystemsmayuseoptimisations(e.g.sparsefiles)sothattheactualphysicalfilesizeissomewhatdifferent.Directoriesandsub-directoriesmayalsorequirespaceinthefilesystem,butthisspaceisnotreportedinthefileSizeresults.
Thatmeansthatthefiguresreporteddonotreflecttheactualdiskusageofthecollectionwith100%accuracy.TheactualdiskusageofacollectionisnormallyslightlyhigherthanthesumofthereportedfileSizevalues.StillthesumofthefileSizevaluescanstillbeusedasalowerboundapproximationofthediskusage.
Examples
arangosh>db.demo.figures()
showexecutionresults
Load
loadsacollectioncollection.load()
Loadsacollectionintomemory.
Note:clustercollectionsareloadedatalltimes.
Examples
arangosh>col=db.example;
CollectionMethods
50
[ArangoCollection16923,"example"(typedocument,statusloaded)]
arangosh>col.load();
arangosh>col;
[ArangoCollection16923,"example"(typedocument,statusloaded)]
Revision
returnstherevisionidofacollectioncollection.revision()
Returnstherevisionidofthecollection
Therevisionidisupdatedwhenthedocumentdataismodified,eitherbyinserting,deleting,updatingorreplacingdocumentsinit.
Therevisionidofacollectioncanbeusedbyclientstocheckwhetherdatainacollectionhaschangedorifitisstillunmodifiedsinceapreviousfetchoftherevisionid.
Therevisionidreturnedisastringvalue.Clientsshouldtreatthisvalueasanopaquestring,andonlyuseitforequality/non-equalitycomparisons.
Path
returnsthephysicalpathofthecollectioncollection.path()
Thepathoperationreturnsastringwiththephysicalstoragepathforthecollectiondata.
Note:thismethodwillreturnnothingmeaningfulinacluster.Inasingle-serverArangoDB,thismethodwillonlyreturnmeaningfuldatafortheMMFilesengine.
Checksum
calculatesachecksumforthedatainacollectioncollection.checksum(withRevisions,withData)
Thechecksumoperationcalculatesanaggregatehashvalueforalldocumentkeyscontainedincollectioncollection.
IftheoptionalargumentwithRevisionsissettotrue,thentherevisionidsofthedocumentsarealsoincludedinthehashcalculation.
IftheoptionalargumentwithDataissettotrue,thenalluser-defineddocumentattributesarealsochecksummed.Includingthedocumentdatainchecksummingwillmakethecalculationslower,butismoreaccurate.
ThechecksumcalculationalgorithmchangedinArangoDB3.0,sochecksumsfrom3.0andearlierversionsforthesamedatawilldiffer.
Note:thismethodisnotavailableinacluster.
Unload
unloadsacollectioncollection.unload()
Startsunloadingacollectionfrommemory.Notethatunloadingisdeferreduntilallqueryhavefinished.
Note:clustercollectionscannotbeunloaded.
Examples
arangosh>col=db.example;
[ArangoCollection7464,"example"(typedocument,statusloaded)]
arangosh>col.unload();
arangosh>col;
[ArangoCollection7464,"example"(typedocument,statusunloaded)]
Rename
renamesacollectioncollection.rename(new-name)
CollectionMethods
51
Renamesacollectionusingthenew-name.Thenew-namemustnotalreadybeusedforadifferentcollection.new-namemustalsobeavalidcollectionname.Formoreinformationonvalidcollectionnamespleaserefertothenamingconventions.
Ifrenamingfailsforanyreason,anerroristhrown.Ifrenamingthecollectionsucceeds,thenthecollectionisalsorenamedinallgraphdefinitionsinsidethe_graphscollectioninthecurrentdatabase.
Note:thismethodisnotavailableinacluster.
Examples
arangosh>c=db.example;
[ArangoCollection17041,"example"(typedocument,statusloaded)]
arangosh>c.rename("better-example");
arangosh>c;
[ArangoCollection17041,"better-example"(typedocument,statusloaded)]
Rotate
rotatesthecurrentjournalofacollectioncollection.rotate()
Rotatesthecurrentjournalofacollection.Thisoperationmakesthecurrentjournalofthecollectionaread-onlydatafilesoitmaybecomeacandidateforgarbagecollection.Ifthereiscurrentlynojournalavailableforthecollection,theoperationwillfailwithanerror.
Note:thismethodisspecificfortheMMFilesstorageengine,andthereitisnotavailableinacluster.
Note:pleasenotethatyouneedappropriateuserpermissionstoexecutethis.
TodotherenamecollectionsinfirstplaceyouneedtohaveadministrativerightsonthedatabaseTohaveaccesstotheresultingrenamedcollectionyoueitherneedtohaveaccesstoallcollectionsofthatdatabase(*)oramainsystemadministratorhastogiveyouaccesstothenewlynamedone.
CollectionMethods
52
DatabaseMethods
Collection
returnsasinglecollectionornulldb._collection(collection-name)
Returnsthecollectionwiththegivennameornullifnosuchcollectionexists.
db._collection(collection-identifier)
Returnsthecollectionwiththegivenidentifierornullifnosuchcollectionexists.Accessingcollectionsbyidentifierisdiscouragedforendusers.Endusersshouldaccesscollectionsusingthecollectionname.
Examples
Getacollectionbyname:
arangosh>db._collection("demo");
[ArangoCollection92,"demo"(typedocument,statusloaded)]
Getacollectionbyid:
arangosh>db._collection(123456);
[ArangoCollection123456,"demo"(typedocument,statusloaded)]
Unknowncollection:
arangosh>db._collection("unknown");
null
Create
createsanewdocumentoredgecollectiondb._create(collection-name)
Createsanewdocumentcollectionnamedcollection-name.Ifthecollectionnamealreadyexistsorifthenameformatisinvalid,anerroristhrown.Formoreinformationonvalidcollectionnamespleaserefertothenamingconventions.
db._create(collection-name,properties)
propertiesmustbeanobjectwiththefollowingattributes:
waitForSync(optional,defaultfalse):Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize(optional,defaultisaconfigurationparameter:Themaximalsizeofajournalordatafile.Notethatthisalsolimitsthemaximalsizeofasingleobject.Mustbeatleast1MB.
isSystem(optional,defaultisfalse):Iftrue,createasystemcollection.Inthiscasecollection-nameshouldstartwithanunderscore.Endusersshouldnormallycreatenon-systemcollectionsonly.APIimplementorsmayberequiredtocreatesystemcollectionsinveryspecialoccasions,butnormallyaregularcollectionwilldo.
isVolatile(optional,defaultisfalse):Iftruethenthecollectiondataiskeptin-memoryonlyandnotmadepersistent.Unloadingthecollectionwillcausethecollectiondatatobediscarded.Stoppingorre-startingtheserverwillalsocausefulllossofdatainthecollection.Thecollectionitselfwillremainhowever(onlythedataisvolatile).SettingthisoptionwillmaketheresultingcollectionbeslightlyfasterthanregularcollectionsbecauseArangoDBdoesnotenforceanysynchronizationtodiskanddoesnotcalculateanyCRCchecksumsfordatafiles(astherearenodatafiles).ThisoptionismeaningfulfortheMMFilesstorageengineonly.
keyOptions(optional):additionaloptionsforkeygeneration.Ifspecified,thenkeyOptionsshouldbeaJSONobjectcontainingthefollowingattributes(note:someofthemareoptional):
type:specifiesthetypeofthekeygenerator.Thecurrentlyavailablegeneratorsaretraditionalandautoincrement.(note:autoincrementiscurrentlyonlysupportedfornon-shardedcollections)
DatabaseMethods
53
allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeofdocumentsisconsideredanerror.increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
numberOfShards(optional,defaultis1):inacluster,thisvaluedeterminesthenumberofshardstocreateforthecollection.Inasingleserversetup,thisoptionismeaningless.
shardKeys(optional,defaultis["_key"]):inacluster,thisattributedetermineswhichdocumentattributesareusedtodeterminethetargetshardfordocuments.Documentsaresenttoshardsbasedonthevaluestheyhaveintheirshardkeyattributes.Thevaluesofallshardkeyattributesinadocumentarehashed,andthehashvalueisusedtodeterminethetargetshard.Notethatvaluesofshardkeyattributescannotbechangedonceset.Thisoptionismeaninglessinasingleserversetup.
Whenchoosingtheshardkeys,onemustbeawareofthefollowingrulesandlimitations:InashardedcollectionwithmorethanonesharditisnotpossibletosetupauniqueconstraintonanattributethatisnottheoneandonlyshardkeygiveninshardKeys.Thisisbecauseenforcingauniqueconstraintwouldotherwisemakeaglobalindexnecessaryorneedextensivecommunicationforeverysinglewriteoperation.Furthermore,if_keyisnottheoneandonlyshardkey,thenitisnotpossibletosetthe_keyattributewheninsertingadocument,providedthecollectionhasmorethanoneshard.Again,thisisbecausethedatabasehastoenforcetheuniqueconstraintonthe_keyattributeandthiscanonlybedoneefficientlyifthisistheonlyshardkeybydelegatingtotheindividualshards.
replicationFactor(optional,defaultis1):inacluster,thisattributedetermineshowmanycopiesofeachshardarekeptondifferentDBServers.Thevalue1meansthatonlyonecopy(nosynchronousreplication)iskept.Avalueofkmeansthatk-1replicasarekept.AnytwocopiesresideondifferentDBServers.Replicationbetweenthemissynchronous,thatis,everywriteoperationtothe"leader"copywillbereplicatedtoall"follower"replicas,beforethewriteoperationisreportedsuccessful.
Ifaserverfails,thisisdetectedautomaticallyandoneoftheserversholdingcopiestakeover,usuallywithoutanerrorbeingreported.
distributeShardsLikedistributetheshardsofthiscollectioncloningthesharddistributionofanother.
WhenusingtheEnterpriseversionofArangoDBthereplicationFactormaybesetto"satellite"makingthecollectionlocallyjoinableoneverydatabaseserver.ThisreducesthenumberofnetworkhopsdramaticallywhenusingjoinsinAQLatthecostsofreducedwriteperformanceonthesecollections.
db._create(collection-name,properties,type)
Specifiestheoptionaltypeofthecollection,itcaneitherbedocumentoredge.Ondefaultitisdocument.Insteadofgivingatypeyoucanalsousedb._createEdgeCollectionordb._createDocumentCollection.
db._create(collection-name,properties[,type],options)
Asanoptionalthird(ifthetypestringisbeingomitted)orfourthparameteryoucanspecifyanoptionaloptionsmapthatcontrolshowtheclusterwillcreatethecollection.Theseoptionsareonlyrelevantatcreationtimeandwillnotbepersisted:
waitForSyncReplication(default:true)Whenenabledtheserverwillonlyreportsuccessbacktotheclientifallreplicashavecreatedthecollection.Settofalseifyouwantfasterserverresponsesanddon'tcareaboutfullreplication.
enforceReplicationFactor(default:true)Whenenabledwhichmeanstheserverwillcheckifthereareenoughreplicasavailableatcreationtimeandbailoutotherwise.Settofalsetodisablethisextracheck.
Examples
Withdefaults:
arangosh>c=db._create("users");
arangosh>c.properties();
showexecutionresultsWithproperties:
arangosh>c=db._create("users",{waitForSync:true,
........>journalSize:1024*1204});
DatabaseMethods
54
arangosh>c.properties();
showexecutionresultsWithakeygenerator:
arangosh>db._create("users",
........>{keyOptions:{type:"autoincrement",offset:10,increment:5}});
arangosh>db.users.save({name:"user1"});
arangosh>db.users.save({name:"user2"});
arangosh>db.users.save({name:"user3"});
showexecutionresultsWithaspecialkeyoption:
arangosh>db._create("users",{keyOptions:{allowUserKeys:false}});
arangosh>db.users.save({name:"user1"});
arangosh>db.users.save({name:"user2",_key:"myuser"});
arangosh>db.users.save({name:"user3"});
showexecutionresultscreatesanewedgecollectiondb._createEdgeCollection(collection-name)
Createsanewedgecollectionnamedcollection-name.Ifthecollectionnamealreadyexistsanerroristhrown.ThedefaultvalueforwaitForSyncisfalse.
db._createEdgeCollection(collection-name,properties)
propertiesmustbeanobjectwiththefollowingattributes:
waitForSync(optional,defaultfalse):Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize(optional,defaultis"configurationparameter"):Themaximalsizeofajournalordatafile.Notethatthisalsolimitsthemaximalsizeofasingleobjectandmustbeatleast1MB.
createsanewdocumentcollectiondb._createDocumentCollection(collection-name)
Createsanewdocumentcollectionnamedcollection-name.Ifthedocumentnamealreadyexistsanderroristhrown.
AllCollections
returnsallcollectionsdb._collections()
Returnsallcollectionsofthegivendatabase.
Examples
arangosh>db._collections();
showexecutionresults
CollectionName
selectsacollectionfromthevocbasedb.collection-name
Returnsthecollectionwiththegivencollection-name.Ifnosuchcollectionexists,createacollectionnamedcollection-namewiththedefaultproperties.
Examples
arangosh>db.example;
[ArangoCollection16607,"example"(typedocument,statusloaded)]
DatabaseMethods
55
Drop
dropsacollectiondb._drop(collection)
Dropsacollectionandallitsindexesanddata.
db._drop(collection-identifier)
Dropsacollectionidentifiedbycollection-identifierwithallitsindexesanddata.Noerroristhrownifthereisnosuchcollection.
db._drop(collection-name)
Dropsacollectionnamedcollection-nameandallitsindexes.Noerroristhrownifthereisnosuchcollection.
db._drop(collection-name,options)
Inordertodropasystemcollection,onemustspecifyanoptionsobjectwithattributeisSystemsettotrue.Otherwiseitisnotpossibletodropsystemcollections.
Note:clustercollection,whichareprototypesforcollectionswithdistributeShardsLikeparameter,cannotbedropped.
Examples
Dropsacollection:
arangosh>col=db.example;
[ArangoCollection16702,"example"(typedocument,statusloaded)]
arangosh>db._drop(col);
arangosh>col;
[ArangoCollection16702,"example"(typedocument,statusloaded)]
Dropsacollectionidentifiedbyname:
arangosh>col=db.example;
[ArangoCollection16713,"example"(typedocument,statusloaded)]
arangosh>db._drop("example");
arangosh>col;
[ArangoCollection16713,"example"(typedocument,statusdeleted)]
Dropsasystemcollection
arangosh>col=db._example;
[ArangoCollection16724,"_example"(typedocument,statusloaded)]
arangosh>db._drop("_example",{isSystem:true});
arangosh>col;
[ArangoCollection16724,"_example"(typedocument,statusdeleted)]
Truncate
truncatesacollectiondb._truncate(collection)
Truncatesacollection,removingalldocumentsbutkeepingallitsindexes.
db._truncate(collection-identifier)
Truncatesacollectionidentifiedbycollection-identified.Noerroristhrownifthereisnosuchcollection.
db._truncate(collection-name)
Truncatesacollectionnamedcollection-name.Noerroristhrownifthereisnosuchcollection.
Examples
Truncatesacollection:
DatabaseMethods
56
arangosh>col=db.example;
arangosh>col.save({"Hello":"World"});
arangosh>col.count();
arangosh>db._truncate(col);
arangosh>col.count();
showexecutionresultsTruncatesacollectionidentifiedbyname:
arangosh>col=db.example;
arangosh>col.save({"Hello":"World"});
arangosh>col.count();
arangosh>db._truncate("example");
arangosh>col.count();
showexecutionresults
DatabaseMethods
57
DocumentsThisisanintroductiontoArangoDB'sinterfaceforworkingwithdocumentsfromtheJavaScriptshellarangoshorinJavaScriptcodeintheserver.ForotherlanguagesseethecorrespondinglanguageAPI.
BasicsandTerminology:sectiononthebasicapproachCollectionMethods:detailedAPIdescriptionforcollectionobjectsDatabaseMethods:detailedAPIdescriptionfordatabaseobjects
Documents
58
BasicsandTerminologyDocumentsinArangoDBareJSONobjects.Theseobjectscanbenested(toanydepth)andmaycontainlists.Eachdocumenthasauniqueprimarykeywhichidentifiesitwithinitscollection.Furthermore,eachdocumentisuniquelyidentifiedbyitsdocumenthandleacrossallcollectionsinthesamedatabase.Differentrevisionsofthesamedocument(identifiedbyitshandle)canbedistinguishedbytheirdocumentrevision.Anytransactiononlyeverseesasinglerevisionofadocument.Forexample:
{
"_id":"myusers/3456789",
"_key":"3456789",
"_rev":"14253647",
"firstName":"John",
"lastName":"Doe",
"address":{
"street":"RoadToNowhere1",
"city":"Gotham"
},
"hobbies":[
{name:"swimming",howFavorite:10},
{name:"biking",howFavorite:6},
{name:"programming",howFavorite:4}
]
}
Alldocumentscontainspecialattributes:thedocumenthandleisstoredasastringin_id,thedocument'sprimarykeyin_keyandthedocumentrevisionin_rev.Thevalueofthe_keyattributecanbespecifiedbytheuserwhencreatingadocument._idand_keyvaluesareimmutableoncethedocumenthasbeencreated.The_revvalueismaintainedbyArangoDBautomatically.
DocumentHandle
Adocumenthandleuniquelyidentifiesadocumentinthedatabase.Itisastringandconsistsofthecollection'snameandthedocumentkey(_keyattribute)separatedby/.
DocumentKey
Adocumentkeyuniquelyidentifiesadocumentinthecollectionitisstoredin.Itcanandshouldbeusedbyclientswhenspecificdocumentsarequeried.Thedocumentkeyisstoredinthe_keyattributeofeachdocument.ThekeyvaluesareautomaticallyindexedbyArangoDBinacollection'sprimaryindex.Thuslookingupadocumentbyitskeyisafastoperation.The_keyvalueofadocumentisimmutableoncethedocumenthasbeencreated.Bydefault,ArangoDBwillauto-generateadocumentkeyifno_keyattributeisspecified,andusetheuser-specified_keyotherwise.Thegenerated_keyisguaranteedtobeuniqueinthecollectionitwasgeneratedfor.Thisalsoappliestoshardedcollectionsinacluster.Itcan'tbeguaranteedthatthe_keyisuniquewithinadatabaseoracrossawholenodeorinstancehowever.
Thisbehaviorcanbechangedonaper-collectionlevelbycreatingcollectionswiththekeyOptionsattribute.
UsingkeyOptionsitispossibletodisallowuser-specifiedkeyscompletely,ortoforceaspecificregimeforauto-generatingthe_keyvalues.
DocumentRevision
AsArangoDBsupportsMVCC(MultipleVersionConcurrencyControl),documentscanexistinmorethanonerevision.ThedocumentrevisionistheMVCCtokenusedtospecifyaparticularrevisionofadocument(identifiedbyits_id).Itisastringvaluethatcontained(uptoArangoDB3.0)anintegernumberandisuniquewithinthelistofdocumentrevisionsforasingledocument.InArangoDB>=3.1the_revstringsareinfacttimestamps.TheyusethelocalclockoftheDBserverthatactuallywritesthedocumentandhavemillisecondaccuracy.Actually,a"HybridLogicalClock"isused(forthisconceptseethispaper).
Withinonesharditisguaranteedthattwodifferentdocumentrevisionshaveadifferent_revstring,eveniftheyarewritteninthesamemillisecond,andthatthesestampsareascending.
BasicsandTerminology
59
Notehoweverthatdifferentserversinyourclustermighthaveaclockskew,andthereforebetweendifferentshardsorevenbetweendifferentcollectionsthetimestampsarenotguaranteedtobecomparable.
TheHybridLogicalClockfeaturedoesonethingtoaddressthisissue:WheneveramessageissentfromsomeserverAinyourclustertoanotheroneB,itisensuredthatanytimestamptakenonBafterthemessagehasarrivedisgreaterthananytimestamptakenonAbeforethemessagewassent.Thisensuresthatifthereissome"causality"betweeneventsondifferentservers,timestampsincreasefromcausetoeffect.Adirectconsequenceofthisisthatsometimesaserverhastotaketimestampsthatseemtocomefromthefutureofitsownclock.Itwillhoweverstillproduceeverincreasingtimestamps.Iftheclockskewissmall,thenyourtimestampswillrelativelyaccuratelydescribethetimewhenthedocumentrevisionwasactuallywritten.
ArangoDBuses64bitunsignedintegervaluestomaintaindocumentrevisionsinternally.Atthisstageweintentionallydonotdocumenttheexactformatoftherevisionvalues.Whenreturningdocumentrevisionstoclients,ArangoDBwillputthemintoastringtoensuretherevisionisnotclippedbyclientsthatdonotsupportbigintegers.ClientsshouldtreattherevisionreturnedbyArangoDBasanopaquestringwhentheystoreoruseitlocally.ThiswillallowArangoDBtochangetheformatofrevisionslaterifthisshouldberequired(ashashappenedwith3.1withtheHybridLogicalClock).Clientscanuserevisionstoperformsimpleequality/non-equalitycomparisons(e.g.tocheckwhetheradocumenthaschangedornot),buttheyshouldnotuserevisionidstoperformgreater/lessthancomparisonswiththemtocheckifadocumentrevisionisolderthanoneanother,evenifthismightworkforsomecases.
Documentrevisionscanbeusedtoconditionallyquery,update,replaceordeletedocumentsinthedatabase.Inordertofindaparticularrevisionofadocument,youneedthedocumenthandleorkey,andthedocumentrevision.
MultipleDocumentsinasingleCommand
BeginningwithArangoDB3.0thebasicdocumentAPIhasbeenextendedtohandlenotonlysingledocumentsbutmultipledocumentsinasinglecommand.Thisiscrucialforperformance,inparticularintheclustersituation,inwhichasinglerequestcaninvolvemultiplenetworkhopswithinthecluster.Anotheradvantageisthatitreducestheoverheadofindividualnetworkroundtripsbetweentheclientandtheserver.ThegeneralideatoperformmultipledocumentoperationsinasinglecommandistouseJSONarraysofobjectsintheplaceofasingledocument.Asaconsequence,documentkeys,handlesandrevisionsforpreconditionshavetobesuppliedembeddedintheindividualdocumentsgiven.Multipledocumentoperationsarerestrictedtoasingledocumentoredgecollection.SeetheAPIdescriptionsforcollectionobjectsfordetails.NotethattheAPIfordatabaseobjectsdonotoffertheseoperations.
BasicsandTerminology
60
CollectionMethods
All
collection.all()
Fetchesalldocumentsfromacollectionandreturnsacursor.YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Examples
UsetoArraytogetalldocumentsatonce:
arangosh>db.five.save({name:"one"});
arangosh>db.five.save({name:"two"});
arangosh>db.five.save({name:"three"});
arangosh>db.five.save({name:"four"});
arangosh>db.five.save({name:"five"});
arangosh>db.five.all().toArray();
showexecutionresultsUselimittorestrictthedocuments:
arangosh>db.five.save({name:"one"});
arangosh>db.five.save({name:"two"});
arangosh>db.five.save({name:"three"});
arangosh>db.five.save({name:"four"});
arangosh>db.five.save({name:"five"});
arangosh>db.five.all().limit(2).toArray();
showexecutionresults
Querybyexamplecollection.byExample(example)
Fetchesalldocumentsfromacollectionthatmatchthespecifiedexampleandreturnsacursor.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.Ifyouuse
{"a":{"c":1}}
asexample,thenyouwillfindalldocuments,suchthattheattributeacontainsadocumentoftheform{c:1}.Forexamplethedocument
{"a":{"c":1},"b":1}
willmatch,butthedocument
{"a":{"c":1,"b":1}}
willnot.
However,ifyouuse
{"a.c":1}
CollectionMethods
61
thenyouwillfindalldocuments,whichcontainasub-documentinathathasanattributecofvalue1.Boththefollowingdocuments
{"a":{"c":1},"b":1}
and
{"a":{"c":1,"b":1}}
willmatch.
collection.byExample(path1,value1,...)
Asalternativeyoucansupplyanarrayofpathsandvalues.
Examples
UsetoArraytogetalldocumentsatonce:
arangosh>db.users.save({name:"Gerhard"});
arangosh>db.users.save({name:"Helmut"});
arangosh>db.users.save({name:"Angela"});
arangosh>db.users.all().toArray();
arangosh>db.users.byExample({"_id":"users/20"}).toArray();
arangosh>db.users.byExample({"name":"Gerhard"}).toArray();
arangosh>db.users.byExample({"name":"Helmut","_id":"users/15"}).toArray();
showexecutionresultsUsenexttoloopoveralldocuments:
arangosh>db.users.save({name:"Gerhard"});
arangosh>db.users.save({name:"Helmut"});
arangosh>db.users.save({name:"Angela"});
arangosh>vara=db.users.byExample({"name":"Angela"});
arangosh>while(a.hasNext())print(a.next());
showexecutionresults
FirstExample
collection.firstExample(example)
Returnssomedocumentofacollectionthatmatchesthespecifiedexample.Ifnosuchdocumentexists,nullwillbereturned.Theexamplehastobespecifiedaspathsandvalues.SeebyExamplefordetails.
collection.firstExample(path1,value1,...)
Asalternativeyoucansupplyanarrayofpathsandvalues.
Examples
arangosh>db.users.firstExample("name","Angela");
showexecutionresults
Range
collection.range(attribute,left,right)
CollectionMethods
62
Returnsalldocumentsfromacollectionsuchthattheattributeisgreaterorequalthanleftandstrictlylessthanright.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.
Note:therangesimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionwithinaspecificrangeistouseanAQLqueryasfollows:
FORdocIN@@collection
FILTERdoc.value>=@left&&doc.value<@right
LIMIT@skip,@limit
RETURNdoc
Examples
UsetoArraytogetalldocumentsatonce:
arangosh>db.old.ensureIndex({type:"skiplist",fields:["age"]});
arangosh>db.old.save({age:15});
arangosh>db.old.save({age:25});
arangosh>db.old.save({age:30});
arangosh>db.old.range("age",10,30).toArray();
showexecutionresults
Closedrange
collection.closedRange(attribute,left,right)
Returnsalldocumentsofacollectionsuchthattheattributeisgreaterorequalthanleftandlessorequalthanright.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.
Note:theclosedRangesimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionwithinaspecificrangeistouseanAQLqueryasfollows:
FORdocIN@@collection
FILTERdoc.value>=@left&&doc.value<=@right
LIMIT@skip,@limit
RETURNdoc
Examples
UsetoArraytogetalldocumentsatonce:
arangosh>db.old.ensureIndex({type:"skiplist",fields:["age"]});
arangosh>db.old.save({age:15});
arangosh>db.old.save({age:25});
arangosh>db.old.save({age:30});
arangosh>db.old.closedRange("age",10,30).toArray();
showexecutionresults
Any
collection.any()
Returnsarandomdocumentfromthecollectionornullifnoneexists.
Note:thismethodisexpensivewhenusingtheRocksDBstorageengine.
CollectionMethods
63
Count
collection.count()
Returnsthenumberoflivingdocumentsinthecollection.
Examples
arangosh>db.users.count();
0
toArray
collection.toArray()
Convertsthecollectionintoanarrayofdocuments.NeverusethiscallinaproductionenvironmentasitwillbasicallycreateacopyofyourcollectioninRAMwhichwilluseresourcesdependingonthenumberandsizeofthedocumentsinyourcollecion.
Documentcollection.document(object)
Thedocumentmethodfindsadocumentgivenanobjectobjectcontainingthe_idor_keyattribute.Themethodreturnsthedocumentifitcanbefound.Ifbothattributesaregiven,the_idtakesprecedence,itisanerror,ifthecollectionpartofthe_iddoesnotmatchthecollection.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.Anerrorisalsothrownifnodocumentexistswiththegiven_idor_keyvalue.
Pleasenotethatifthemethodisexecutedonthearangodserver(e.g.frominsideaFoxxapplication),animmutabledocumentobjectwillbereturnedforperformancereasons.Itisnotpossibletochangeattributesofthisimmutableobject.Toupdateorpatchthereturneddocument,itneedstobecloned/copiedintoaregularJavaScriptobjectfirst.Thisisnotnecessaryifthedocumentmethodiscalledfromoutofarangoshorfromanyotherclient.
collection.document(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.Norevisioncanbespecifiedinthiscase.
collection.document(document-key)
Asbefore.Insteadofobjectadocument-keycanbepassedasfirstargument.
collection.document(array)
Thisvariantallowstoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifdocumentwouldhavebeencalledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.
Examples
Returnsthedocumentforadocument-handle:
arangosh>db.example.document("example/2873916");
showexecutionresultsReturnsthedocumentforadocument-key:
arangosh>db.example.document("2873916");
showexecutionresultsReturnsthedocumentforanobject:
arangosh>db.example.document({_id:"example/2873916"});
CollectionMethods
64
showexecutionresultsReturnsthedocumentforanarrayoftwokeys:
arangosh>db.example.document(["2873916","2873917"]);
showexecutionresultsAnerrorisraisedifthedocumentisunknown:
arangosh>db.example.document("example/4472917");
[ArangoError1202:documentnotfound]
Anerrorisraisedifthehandleisinvalid:
arangosh>db.example.document("");
[ArangoError1205:illegaldocumenthandle]
Changesin3.0from2.8:
documentcannowquerymultipledocumentswithonecall.
Exists
checkswhetheradocumentexistscollection.exists(object)
Theexistsmethoddetermineswhetheradocumentexistsgivenanobjectobjectcontainingthe_idor_keyattribute.Ifbothattributesaregiven,the_idtakesprecedence,itisanerror,ifthecollectionpartofthe_iddoesnotmatchthecollection.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.
Insteadofreturningthefounddocumentoranerror,thismethodwillonlyreturnanobjectwiththeattributes_id,_keyand_rev,orfalseifnodocumentwiththegiven_idor_keyexists.Itcanthusbeusedforeasyexistencechecks.
Thismethodwillthrowanerrorifusedimproperly,e.g.whencalledwithanon-documenthandle,anon-document,orwhenacross-collectionrequestisperformed.
collection.exists(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.
collection.exists(document-key)
Asbefore.Insteadofobjectadocument-keycanbepassedasfirstargument.
collection.exists(array)
Thisvariantallowstoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifexistswouldhavebeencalledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,theoperationstopsimmediatelyreturningonlyanerrorobject.
Changesin3.0from2.8:
Inthecaseofarevisionmismatchexistsnowthrowsanerrorinsteadofsimplyreturningfalse.Thisistomakeitpossibletotellthedifferencebetweenarevisionmismatchandanon-existingdocument.
existscannowquerymultipledocumentswithonecall.
LookupByKeyscollection.documents(keys)
CollectionMethods
65
Looksupthedocumentsinthespecifiedcollectionusingthearrayofkeysprovided.Alldocumentsforwhichamatchingkeywasspecifiedinthekeysarrayandthatexistinthecollectionwillbereturned.Keysforwhichnodocumentcanbefoundintheunderlyingcollectionareignored,andnoexceptionwillbethrownforthem.
Thismethodisdeprecatedinfavourofthearrayvariantofdocument.
Examples
arangosh>keys=[];
arangosh>for(vari=0;i<10;++i){
........>db.example.insert({_key:"test"+i,value:i});
........>keys.push("test"+i);
........>}
arangosh>db.example.documents(keys);
showexecutionresults
Insert
collection.insert(data)
Createsanewdocumentinthecollectionfromthegivendata.Thedatamustbeanobject.Theattributes_idand_revareignoredandareautomaticallygenerated.Auniquevaluefortheattribute_keywillbeautomaticallygeneratedifnotspecified.Ifspecified,theremustnotbeadocumentwiththegiven_keyinthecollection.
Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleofthenewlycreateddocument,theattribute_keythedocumentkeyandtheattribute_revcontainsthedocumentrevision.
collection.insert(data,options)
Createsanewdocumentinthecollectionfromthegivendataasabove.Theoptionaloptionsparametermustbeanobjectandcanbeusedtospecifythefollowingoptions:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.silent:Ifthisflagissettotrue,themethoddoesnotreturnanyoutput.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.
Note:sinceArangoDB2.2,insertisanaliasforsave.
collection.insert(array)
collection.insert(array,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifinsertwouldhavebeencalledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.
Changesin3.0from2.8:
TheoptionssilentandreturnNewarenew.Themethodcannowinsertmultipledocumentswithonecall.
Examples
arangosh>db.example.insert({Hello:"World"});
arangosh>db.example.insert({Hello:"World"},{waitForSync:true});
showexecutionresults
arangosh>db.example.insert([{Hello:"World"},{Hello:"there"}])
CollectionMethods
66
arangosh>db.example.insert([{Hello:"World"},{}],{waitForSync:true});
showexecutionresults
Replace
collection.replace(selector,data)
Replacesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbeadocumentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenreplacedwiththedatagivenassecondargument.Anyattribute_id,_keyor_revindataisignored.
Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleoftheupdateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainstherevisionoftheold(nowreplaced)document.
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,thereisaconflict,andanerroristhrown.
collection.replace(selector,data,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.
collection.replace(document-handle,data)
collection.replace(document-handle,data,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.
collection.replace(document-key,data)
collection.replace(document-key,data,options)
Asbefore.Insteadofselectoradocument-keycanbepassedasfirstargument.Norevisionpreconditionistested.
collection.replace(selectorarray,dataarray)
collection.replace(selectorarray,dataarray,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofselector/datapairs.Thetwoarraysgivenasselectorarrayanddataarraymusthavethesamelength.Thebehaviorisexactlyasifreplacewouldhavebeencalledonallrespectivemembersofthetwoarraysandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.
Examples
Createandupdateadocument:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db.example.replace(a1,{a:2});
arangosh>a3=db.example.replace(a1,{a:3});
showexecutionresultsUseadocumenthandle:
arangosh>a1=db.example.insert({a:1});
CollectionMethods
67
arangosh>a2=db.example.replace("example/3903044",{a:2});
showexecutionresults
Changesin3.0from2.8:
Theoptionssilent,returnNewandreturnOldarenew.Themethodcannowreplacemultipledocumentswithonecall.
Update
collection.update(selector,data)
Updatesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbeadocumentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenpatchedwiththedatagivenassecondargument.Anyattribute_id,_keyor_revindataisignored.
Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleoftheupdateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainstherevisionoftheold(nowupdated)document.
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,thereisaconflict,andanerroristhrown.
collection.update(selector,data,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.keepNull:TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesarestoredinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnullvalueswillberemovedfromthetargetdocument.mergeObjects:Controlswhetherobjects(notarrays)willbemergedifpresentinboththeexistingandthepatchdocument.Ifsettofalse,thevalueinthepatchdocumentwilloverwritetheexistingdocument'svalue.Ifsettotrue,objectswillbemerged.Thedefaultistrue.
collection.update(document-handle,data)
collection.update(document-handle,data,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.
collection.update(document-key,data)
collection.update(document-key,data,options)
Asbefore.Insteadofselectoradocument-keycanbepassedasfirstargument.Norevisionpreconditionistested.
collection.update(selectorarray,dataarray)
collection.update(selectorarray,dataarray,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofselector/datapairs.Thetwoarraysgivenasselectorarrayanddataarraymusthavethesamelength.Thebehaviorisexactlyasifupdatewouldhavebeencalledonallrespectivemembersofthetwoarraysandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.
Examples
Createandupdateadocument:
CollectionMethods
68
arangosh>a1=db.example.insert({"a":1});
arangosh>a2=db.example.update(a1,{"b":2,"c":3});
arangosh>a3=db.example.update(a1,{"d":4});
arangosh>a4=db.example.update(a2,{"e":5,"f":6});
arangosh>db.example.document(a4);
arangosh>a5=db.example.update(a4,{"a":1,c:9,e:42});
arangosh>db.example.document(a5);
showexecutionresultsUseadocumenthandle:
arangosh>a1=db.example.insert({"a":1});
arangosh>a2=db.example.update("example/18612115",{"x":1,"y":2});
showexecutionresultsUsethekeepNullparametertoremoveattributeswithnullvalues:
arangosh>db.example.insert({"a":1});
arangosh>db.example.update("example/19988371",
........>{"b":null,"c":null,"d":3});
arangosh>db.example.document("example/19988371");
arangosh>db.example.update("example/19988371",{"a":null},false,false);
arangosh>db.example.document("example/19988371");
arangosh>db.example.update("example/19988371",
........>{"b":null,"c":null,"d":null},false,false);
arangosh>db.example.document("example/19988371");
showexecutionresultsPatchingarrayvalues:
arangosh>db.example.insert({"a":{"one":1,"two":2,"three":3},
........>"b":{}});
arangosh>db.example.update("example/20774803",{"a":{"four":4},
........>"b":{"b1":1}});
arangosh>db.example.document("example/20774803");
arangosh>db.example.update("example/20774803",{"a":{"one":null},
........>"b":null},
........>false,false);
arangosh>db.example.document("example/20774803");
showexecutionresults
Changesin3.0from2.8:
Theoptionssilent,returnNewandreturnOldarenew.Themethodcannowupdatemultipledocumentswithonecall.
Remove
collection.remove(selector)
Removesadocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbeadocumentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenremoved.
Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleoftheremoveddocument,theattribute_revcontainsthedocumentrevisionoftheremoveddocument.
CollectionMethods
69
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,thereisaconflict,andanerroristhrown.
collection.remove(selector,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.
collection.remove(document-handle)
collection.remove(document-handle,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisioncheckisperformed.
collection.remove(document-key)
collection.remove(document-handle,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisioncheckisperformed.
collection.remove(selectorarray)
collection.remove(selectorarray,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofselectors.Thebehaviorisexactlyasifremovewouldhavebeencalledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.
Examples
Removeadocument:
arangosh>a1=db.example.insert({a:1});
arangosh>db.example.document(a1);
arangosh>db.example.remove(a1);
arangosh>db.example.document(a1);
showexecutionresultsRemoveadocumentwithaconflict:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db.example.replace(a1,{a:2});
arangosh>db.example.remove(a1);
arangosh>db.example.remove(a1,true);
arangosh>db.example.document(a1);
showexecutionresults
Changesin3.0from2.8:
Themethodnowreturnsnotonlytruebutinformationabouttheremoveddocument(s).TheoptionssilentandreturnOldarenew.Themethodcannowremovemultipledocumentswithonecall.
RemoveByKeys
collection.removeByKeys(keys)
CollectionMethods
70
Looksupthedocumentsinthespecifiedcollectionusingthearrayofkeysprovided,andremovesalldocumentsfromthecollectionwhosekeysarecontainedinthekeysarray.Keysforwhichnodocumentcanbefoundintheunderlyingcollectionareignored,andnoexceptionwillbethrownforthem.
Themethodwillreturnanobjectcontainingthenumberofremoveddocumentsintheremovedsub-attribute,andthenumberofnot-removed/ignoreddocumentsintheignoredsub-attribute.
Thismethodisdeprecatedinfavourofthearrayvariantofremove.
Examples
arangosh>keys=[];
arangosh>for(vari=0;i<10;++i){
........>db.example.insert({_key:"test"+i,value:i});
........>keys.push("test"+i);
........>}
arangosh>db.example.removeByKeys(keys);
showexecutionresults
RemoveByExamplecollection.removeByExample(example)
Removesalldocumentsmatchinganexample.
collection.removeByExample(document,waitForSync)
TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentdeletionoperationtodiskevenincasethatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.
collection.removeByExample(document,waitForSync,limit)
Theoptionallimitparametercanbeusedtorestrictthenumberofremovalstothespecifiedvalue.Iflimitisspecifiedbutlessthanthenumberofdocumentsinthecollection,itisundefinedwhichdocumentsareremoved.
Examples
arangosh>db.example.removeByExample({Hello:"world"});
1
ReplaceByExample
collection.replaceByExample(example,newValue)
Replacesalldocumentsmatchinganexamplewithanewdocumentbody.TheentiredocumentbodyofeachdocumentmatchingtheexamplewillbereplacedwithnewValue.Thedocumentmeta-attributes_id,_keyand_revwillnotbereplaced.
collection.replaceByExample(document,newValue,waitForSync)
TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentreplacementoperationtodiskevenincasethatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.
collection.replaceByExample(document,newValue,waitForSync,limit)
Theoptionallimitparametercanbeusedtorestrictthenumberofreplacementstothespecifiedvalue.Iflimitisspecifiedbutlessthanthenumberofdocumentsinthecollection,itisundefinedwhichdocumentsarereplaced.
CollectionMethods
71
Examples
arangosh>db.example.save({Hello:"world"});
arangosh>db.example.replaceByExample({Hello:"world"},{Hello:"mars"},false,5);
showexecutionresults
UpdateByExamplecollection.updateByExample(example,newValue)
Partiallyupdatesalldocumentsmatchinganexamplewithanewdocumentbody.SpecificattributesinthedocumentbodyofeachdocumentmatchingtheexamplewillbeupdatedwiththevaluesfromnewValue.Thedocumentmeta-attributes_id,_keyand_revcannotbeupdated.
Partialupdatecouldalsobeusedtoappendnewfields,iftherewerenooldfieldwithsamename.
collection.updateByExample(document,newValue,keepNull,waitForSync)
TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesarestoredinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnullvalueswillberemovedfromthetargetdocument.
TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentreplacementoperationtodiskevenincasethatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.
collection.updateByExample(document,newValue,keepNull,waitForSync,limit)
Theoptionallimitparametercanbeusedtorestrictthenumberofupdatestothespecifiedvalue.Iflimitisspecifiedbutlessthanthenumberofdocumentsinthecollection,itisundefinedwhichdocumentsareupdated.
collection.updateByExample(document,newValue,options)
Usingthisvariant,theoptionsfortheoperationcanbepassedusinganobjectwiththefollowingsub-attributes:
keepNullwaitForSynclimitmergeObjects
Examples
arangosh>db.example.save({Hello:"world",foo:"bar"});
arangosh>db.example.updateByExample({Hello:"world"},{Hello:"foo",World:"bar"},
false);
arangosh>db.example.byExample({Hello:"foo"}).toArray()
showexecutionresults
Collectiontypecollection.type()
Returnsthetypeofacollection.Possiblevaluesare:
2:documentcollection3:edgecollection
GettheVersionofArangoDBdb._version()
CollectionMethods
72
Returnstheserverversionstring.Notethatthisisnottheversionofthedatabase.
Examples
arangosh>require("@arangodb").db._version();
3.3.2
Edges
Edgesarenormaldocumentsthatalwayscontaina_fromanda_toattribute.Therefore,youcanusethedocumentmethodstooperateonedges.Thefollowingmethods,however,arespecifictoedges.
edge-collection.edges(vertex)
Theedgesoperatorfindsalledgesstartingfrom(outbound)orendingin(inbound)vertex.
edge-collection.edges(vertices)
Theedgesoperatorfindsalledgesstartingfrom(outbound)orendingin(inbound)adocumentfromvertices,whichmustbealistofdocumentsordocumenthandles.
arangosh>db._create("vertex");
arangosh>db._createEdgeCollection("relation");
arangosh>varmyGraph={};
arangosh>myGraph.v1=db.vertex.insert({name:"vertex1"});
arangosh>myGraph.v2=db.vertex.insert({name:"vertex2"});
arangosh>myGraph.e1=db.relation.insert(myGraph.v1,myGraph.v2,
........>{label:"knows"});
arangosh>db._document(myGraph.e1);
arangosh>db.relation.edges(myGraph.e1._id);
showexecutionresultsedge-collection.inEdges(vertex)
Theedgesoperatorfindsalledgesendingin(inbound)vertex.
edge-collection.inEdges(vertices)
Theedgesoperatorfindsalledgesendingin(inbound)adocumentfromvertices,whichmustalistofdocumentsordocumenthandles.
Examples
arangosh>db._create("vertex");
arangosh>db._createEdgeCollection("relation");
arangosh>myGraph.v1=db.vertex.insert({name:"vertex1"});
arangosh>myGraph.v2=db.vertex.insert({name:"vertex2"});
arangosh>myGraph.e1=db.relation.insert(myGraph.v1,myGraph.v2,
........>{label:"knows"});
arangosh>db._document(myGraph.e1);
arangosh>db.relation.inEdges(myGraph.v1._id);
arangosh>db.relation.inEdges(myGraph.v2._id);
showexecutionresultsedge-collection.outEdges(vertex)
Theedgesoperatorfindsalledgesstartingfrom(outbound)vertices.
edge-collection.outEdges(vertices)
Theedgesoperatorfindsalledgesstartingfrom(outbound)adocumentfromvertices,whichmustalistofdocumentsordocumenthandles.
Examples
CollectionMethods
73
arangosh>db._create("vertex");
arangosh>db._createEdgeCollection("relation");
arangosh>myGraph.v1=db.vertex.insert({name:"vertex1"});
arangosh>myGraph.v2=db.vertex.insert({name:"vertex2"});
arangosh>myGraph.e1=db.relation.insert(myGraph.v1,myGraph.v2,
........>{label:"knows"});
arangosh>db._document(myGraph.e1);
arangosh>db.relation.outEdges(myGraph.v1._id);
arangosh>db.relation.outEdges(myGraph.v2._id);
showexecutionresults
Misc
collection.iterate(iterator,options)
Iteratesoversomeelementsofthecollectionandapplythefunctioniteratortotheelements.Thefunctionwillbecalledwiththedocumentasfirstargumentandthecurrentnumber(startingwith0)assecondargument.
optionsmustbeanobjectwiththefollowingattributes:
limit(optional,defaultnone):useatmostlimitdocuments.
probability(optional,defaultall):anumberbetween0and1.Documentsarechosenwiththisprobability.
Examples
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.example.save({name:"Name/"+i+"/"+j,
........>home:[i,j],
........>work:[-i,-j]});
........>}
........>}
........>
arangosh>db.example.ensureIndex({type:"geo",fields:["home"]});
arangosh>items=db.example.getIndexes().map(function(x){returnx.id;});
........>db.example.index(items[1]);
showexecutionresults
CollectionMethods
74
DatabaseMethods
Document
db._document(object)
The_documentmethodfindsadocumentgivenanobjectobjectcontainingthe_idattribute.Themethodreturnsthedocumentifitcanbefound.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.Anerrorisalsothrownifnodocumentexistswiththegiven_id.
Pleasenotethatifthemethodisexecutedonthearangodserver(e.g.frominsideaFoxxapplication),animmutabledocumentobjectwillbereturnedforperformancereasons.Itisnotpossibletochangeattributesofthisimmutableobject.Toupdateorpatchthereturneddocument,itneedstobecloned/copiedintoaregularJavaScriptobjectfirst.Thisisnotnecessaryifthe_documentmethodiscalledfromoutofarangoshorfromanyotherclient.
db._document(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.Norevisioncanbespecifiedinthiscase.
Examples
Returnsthedocument:
arangosh>db._document("example/12345");
showexecutionresults
Existsdb._exists(object)
The_existsmethoddetermineswhetheradocumentexistsgivenanobjectobjectcontainingthe_idattribute.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.
Insteadofreturningthefounddocumentoranerror,thismethodwillonlyreturnanobjectwiththeattributes_id,_keyand_rev,orfalseifnodocumentwiththegiven_idor_keyexists.Itcanthusbeusedforeasyexistencechecks.
Thismethodwillthrowanerrorifusedimproperly,e.g.whencalledwithanon-documenthandle,anon-document,orwhenacross-collectionrequestisperformed.
db._exists(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.
Changesin3.0from2.8:
Inthecaseofarevisionmismatch_existsnowthrowsanerrorinsteadofsimplyreturningfalse.Thisistomakeitpossibletotellthedifferencebetweenarevisionmismatchandanon-existingdocument.
Replace
db._replace(selector,data)
Replacesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocumentwiththat_idinthecurrentdatabase.Thisdocumentisthenreplacedwiththedatagivenassecondargument.Anyattribute_id,_keyor_revindataisignored.
Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleoftheupdateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainstherevisionoftheold(nowreplaced)document.
DatabaseMethods
75
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,thereisaconflict,andanerroristhrown.
collection.replace(selector,data,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.
db._replace(document-handle,data)
db._replace(document-handle,data,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.
Examples
Createandreplaceadocument:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db._replace(a1,{a:2});
arangosh>a3=db._replace(a1,{a:3});
showexecutionresults
Changesin3.0from2.8:
Theoptionssilent,returnNewandreturnOldarenew.
Update
db._update(selector,data)
Updatesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocumentwiththat_idinthecurrentdatabase.Thisdocumentisthenpatchedwiththedatagivenassecondargument.Anyattribute_id,_keyor_revindataisignored.
Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleoftheupdateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainstherevisionoftheold(nowupdated)document.
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,thereisaconflict,andanerroristhrown.
db._update(selector,data,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.
DatabaseMethods
76
silent:Ifthisflagissettotrue,nooutputisreturned.keepNull:TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesarestoredinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnullvalueswillberemovedfromthetargetdocument.mergeObjects:Controlswhetherobjects(notarrays)willbemergedifpresentinboththeexistingandthepatchdocument.Ifsettofalse,thevalueinthepatchdocumentwilloverwritetheexistingdocument'svalue.Ifsettotrue,objectswillbemerged.Thedefaultistrue.
db._update(document-handle,data)
db._update(document-handle,data,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.
Examples
Createandupdateadocument:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db._update(a1,{b:2});
arangosh>a3=db._update(a1,{c:3});
showexecutionresults
Changesin3.0from2.8:
Theoptionssilent,returnNewandreturnOldarenew.
Removedb._remove(selector)
Removesadocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocumentwiththat_idinthecurrentdatabase.Thisdocumentisthenremoved.
Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleoftheremoveddocument,theattribute_revcontainsthedocumentrevisionoftheremovedeocument.
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,thereisaconflict,andanerroristhrown.
db._remove(selector,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagisbeendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.silent:Ifthisflagissettotrue,nooutputisreturned.
db._remove(document-handle)
db._remove(document-handle,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisioncheckisperformed.
Examples
Removeadocument:
arangosh>a1=db.example.insert({a:1});
DatabaseMethods
77
arangosh>db._remove(a1);
arangosh>db._remove(a1);
arangosh>db._remove(a1,{overwrite:true});
showexecutionresultsRemovethedocumentintherevisiona1withaconflict:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db._replace(a1,{a:2});
arangosh>db._remove(a1);
arangosh>db._remove(a1,{overwrite:true});
arangosh>db._document(a1);
showexecutionresultsRemoveadocumentusingnewsignature:
arangosh>db.example.insert({_key:"11265325374",a:1});
arangosh>db.example.remove("example/11265325374",
........>{overwrite:true,waitForSync:false})
showexecutionresults
Changesin3.0from2.8:
Themethodnowreturnsnotonlytruebutinformationabouttheremoveddocument(s).TheoptionssilentandreturnOldarenew.
DatabaseMethods
78
Graphs,Vertices&EdgesGraphs,vertices&edgesaredefinedintheGraphschapterindetails.
Relatedblogposts:
Graphsindatamodeling-istheemperornaked?IndexFreeAdjacencyorHybridIndexesforGraphDatabases
Graphs,Vertices&Edges
79
NamingConventionsinArangoDBThefollowingnamingconventionsshouldbefollowedbyuserswhencreatingdatabases,collectionsanddocumentsinArangoDB.
NamingConventions
80
DatabaseNamesArangoDBwillalwaysstartupwithadefaultdatabase,named_system.UserscancreateadditionaldatabasesinArangoDB,providedthedatabasenamesconformtothefollowingconstraints:
Databasenamesmustonlyconsistofthelettersatoz(bothloweranduppercaseallowed),thenumbers0to9,andtheunderscore(_)ordash(-)symbolsThisalsomeansthatanynon-ASCIIdatabasenamesarenotallowedDatabasenamesmustalwaysstartwithaletter.Databasenamesstartingwithanunderscoreareconsideredtobesystemdatabases,andusersshouldnotcreateordeletethoseThemaximumallowedlengthofadatabasenameis64bytesDatabasenamesarecase-sensitive
DatabaseNames
81
CollectionNamesUserscanpicknamesfortheircollectionsasdesired,providedthefollowingnamingconstraintsarenotviolated:
Collectionnamesmustonlyconsistofthelettersatoz(bothinloweranduppercase),thenumbers0to9,andtheunderscore(_)ordash(-)symbols.Thisalsomeansthatanynon-ASCIIcollectionnamesarenotallowedUser-definedcollectionnamesmustalwaysstartwithaletter.Systemcollectionnamesmuststartwithanunderscore.AllcollectionnamesstartingwithanunderscoreareconsideredtobesystemcollectionsthatareforArangoDB'sinternaluseonly.SystemcollectionnamesshouldnotbeusedbyendusersfortheirowncollectionsThemaximumallowedlengthofacollectionnameis64bytesCollectionnamesarecase-sensitive
CollectionNames
82
DocumentKeysUserscandefinetheirownkeysfordocumentstheysave.Thedocumentkeywillbesavedalongwithadocumentinthe_keyattribute.Userscanpickkeyvaluesasrequired,providedthatthevaluesconformtothefollowingrestrictions:
Thekeymustbeastringvalue.Numerickeysarenotallowed,butanynumericvaluecanbeputintoastringandcanthenbeusedasdocumentkey.Thekeymustbeatleast1byteandatmost254byteslong.Emptykeysaredisallowedwhenspecified(thoughitmaybevalidtocompletelyomitthe_keyattributefromadocument)Itmustconsistofthelettersa-z(loweroruppercase),thedigits0-9oranyofthefollowingpunctuationcharacters:_-:.@()+,=;$!*'%Anyothercharacters,especiallymulti-byteUTF-8sequences,whitespaceorpunctuationcharacterscannotbeusedinsidekeyvaluesThekeymustbeuniquewithinthecollectionitisused
Keysarecase-sensitive,i.e.myKeyandMyKEYareconsideredtobedifferentkeys.
Specifyingadocumentkeyisoptionalwhencreatingnewdocuments.Ifnodocumentkeyisspecifiedbytheuser,ArangoDBwillcreatethedocumentkeyitselfaseachdocumentisrequiredtohaveakey.
Therearenoguaranteesabouttheformatandpatternofauto-generateddocumentkeysotherthantheaboverestrictions.Clientsshouldthereforetreatauto-generateddocumentkeysasopaquevaluesandnotrelyontheirformat.
Thecurrentformatforgeneratedkeysisastringcontainingnumericdigits.Thenumericvaluesreflectchronologicaltimeinthesensethat_keyvaluesgeneratedlaterwillcontainhighernumbersthan_keyvaluesgeneratedearlier.Buttheexactvaluethatwillbegeneratedbytheserverisnotpredictable.Notethatifyousortonthe_keyattribute,stringcomparisonwillbeused,whichmeans"100"islessthan"99"etc.
DocumentKeys
83
AttributeNamesUserscanpickattributenamesfordocumentattributesasdesired,providedthefollowingattributenamingconstraintsarenotviolated:
AttributenamesstartingwithanunderscoreareconsideredtobesystemattributesforArangoDB'sinternaluse.SuchattributenamesarealreadyusedbyArangoDBforspecialpurposes:
_idisusedtocontainadocument'shandle_keyisusedtocontainadocument'suser-definedkey_revisusedtocontainthedocument'srevisionnumberInedgecollections,the
_from_to
attributesareusedtoreferenceotherdocuments.
Moresystemattributesmaybeaddedinthefuturewithoutfurthernoticesoendusersshouldtrytoavoidusingtheirownattributenamesstartingwithunderscores.
Theoretically,attributenamescanincludepunctuationandspecialcharactersasdesired,providedthenameisavalidUTF-8string.Formaximumportability,specialcharactersshouldbeavoidedthough.Forexample,attributenamesmaycontainthedotsymbol,butthedothasaspecialmeaninginJavaScriptandalsoinAQL,sowhenusingsuchattributenamesinoneoftheselanguages,theattributenameneedstobequotedbytheenduser.Overallitmightbebettertouseattributenameswhichdon'trequireanyquoting/escapinginalllanguagesused.Thisincludeslanguagesusedbytheclient(e.g.Ruby,PHP)iftheattributesaremappedtoobjectmembersthere.
Attributenamesstartingwithanat-mark(@)willneedtobeenclosedinbacktickswhenusedinanAQLquerytotellthemapartfrombindvariables.Thereforewedonotencouragetheuseofattributesstartingwithat-marks,thoughtheywillworkwhenusedproperly.ArangoDBdoesnotenforcealengthlimitforattributenames.However,longattributenamesmayusemorememoryinresultsetsetc.Thereforetheuseoflongattributenamesisdiscouraged.Attributenamesarecase-sensitive.Attributeswithemptynames(anemptystring)aredisallowed.
AttributeNames
84
HandlingIndexesThisisanintroductiontoArangoDB'sinterfaceforindexesingeneral.Therearespecialsectionsfor
IndexBasics:IntroductiontoallindextypesWhichindextousewhen:IndextypeandoptionsadviserIndexUtilization:HowArangoDBusesindexesWorkingwithIndexes:Howtohandleindexesprogrammaticallyusingthedbobject
HashIndexesSkiplistsPersistentIndexesFulltextIndexesGeo-spatialIndexesVertex-centricIndexes
Indexing
85
Indexbasics
Indexesallowfastaccesstodocuments,providedtheindexedattribute(s)areusedinaquery.WhileArangoDBautomaticallyindexessomesystemattributes,usersarefreetocreateextraindexesonnon-systemattributesofdocuments.
User-definedindexescanbecreatedoncollectionlevel.Mostuser-definedindexescanbecreatedbyspecifyingthenamesoftheindexattributes.Someindextypesallowindexingjustoneattribute(e.g.fulltextindex)whereasotherindextypesallowindexingmultipleattributesatthesametime.
Thesystemattributes_id,_key,_fromand_toareautomaticallyindexedbyArangoDB,withouttheuserbeingrequiredtocreateextraindexesforthem._idand_keyarecoveredbyacollection'sprimarykey,and_fromand_toarecoveredbyanedgecollection'sedgeindexautomatically.
Usingthesystemattribute_idinuser-definedindexesisnotpossible,butindexing_key,_rev,_from,and_tois.
ArangoDBprovidesthefollowingindextypes:
PrimaryIndex
Foreachcollectiontherewillalwaysbeaprimaryindexwhichisahashindexforthedocumentkeys(_keyattribute)ofalldocumentsinthecollection.Theprimaryindexallowsquickselectionofdocumentsinthecollectionusingeitherthe_keyor_idattributes.ItwillbeusedfromwithinAQLqueriesautomaticallywhenperformingequalitylookupson_keyor_id.
Therearealsodedicatedfunctionstofindadocumentgivenits_keyor_idthatwillalwaysmakeuseoftheprimaryindex:
db.collection.document("<document-key>");
db._document("<document-id>");
Astheprimaryindexisanunsortedhashindex,itcannotbeusedfornon-equalityrangequeriesorforsorting.
Theprimaryindexofacollectioncannotbedroppedorchanged,andthereisnomechanismtocreateuser-definedprimaryindexes.
EdgeIndex
Everyedgecollectionalsohasanautomaticallycreatededgeindex.Theedgeindexprovidesquickaccesstodocumentsbyeithertheir_fromor_toattributes.Itcanthereforebeusedtoquicklyfindconnectionsbetweenvertexdocumentsandisinvokedwhentheconnectingedgesofavertexarequeried.
EdgeindexesareusedfromwithinAQLwhenperformingequalitylookupson_fromor_tovaluesinanedgecollections.Therearealsodedicatedfunctionstofindedgesgiventheir_fromor_tovaluesthatwillalwaysmakeuseoftheedgeindex:
db.collection.edges("<from-value>");
db.collection.edges("<to-value>");
db.collection.outEdges("<from-value>");
db.collection.outEdges("<to-value>");
db.collection.inEdges("<from-value>");
db.collection.inEdges("<to-value>");
Internally,theedgeindexisimplementedasahashindex,whichstorestheunionofall_fromand_toattributes.Itcanbeusedforequalitylookups,butnotforrangequeriesorforsorting.Edgeindexesareautomaticallycreatedforedgecollections.Itisnotpossibletocreateuser-definededgeindexes.However,itispossibletofreelyusethe_fromand_toattributesinuser-definedindexes.
Anedgeindexcannotbedroppedorchanged.
HashIndex
Ahashindexcanbeusedtoquicklyfinddocumentswithspecificattributevalues.Thehashindexisunsorted,soitsupportsequalitylookupsbutnorangequeriesorsorting.
IndexBasics
86
Ahashindexcanbecreatedononeormultipledocumentattributes.Ahashindexwillonlybeusedbyaqueryifallindexattributesarepresentinthesearchcondition,andifallattributesarecomparedusingtheequality(==)operator.HashindexesareusedfromwithinAQLandseveralqueryfunctions,e.g.byExample,firstExampleetc.
Hashindexescanoptionallybedeclaredunique,thendisallowingsavingthesamevalue(s)intheindexedattribute(s).Hashindexescanoptionallybesparse.
Thedifferenttypesofhashindexeshavethefollowingcharacteristics:
uniquehashindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.Tryingtoinsertadocumentwiththesamekeyvalueasanalreadyexistingdocumentwillleadtoauniqueconstraintviolation.
Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueofnullintheindexattribute(s)willstillbeindexed.Akeyvalueofnullmayonlyoccuronceintheindex,sothistypeofindexcannotbeusedforoptionalattributes.
Theuniqueoptioncanalsobeusedtoensurethatnoduplicateedgesarecreated,byaddingacombinedindexforthefields_fromand_totoanedgecollection.
unique,sparsehashindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.Documentsinwhichatleastoneoftheindexattributesisnotsetorhasavalueofnullarenotincludedintheindex.Thistypeofindexcanbeusedtoensurethattherearenoduplicatekeysinthecollectionfordocumentswhichhavetheindexedattributesset.Astheindexwillexcludedocumentsforwhichtheindexedattributesarenullornotset,itcanbeusedforoptionalattributes.
non-uniquehashindex:alldocumentsinthecollectionwillbeindexed.Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueofnullintheindexattribute(s)willstillbeindexed.Duplicatekeyvaluescanoccuranddonotleadtouniqueconstraintviolations.
non-unique,sparsehashindex:onlythosedocumentswillbeindexedthathavealltheindexedattributessettoavalueotherthannull.Itcanbeusedforoptionalattributes.
Theamortizedcomplexityoflookup,insert,update,andremovaloperationsinuniquehashindexesisO(1).
Non-uniquehashindexeshaveanamortizedcomplexityofO(1)forinsert,update,andremovaloperations.Thatmeansnon-uniquehashindexescanbeusedonattributeswithlowcardinality.
Ifahashindexiscreatedonanattributethatismissinginallormanyofthedocuments,thebehaviorisasfollows:
iftheindexissparse,thedocumentsmissingtheattributewillnotbeindexedandnotuseindexmemory.Thesedocumentswillnotinfluencetheupdateorremovalperformancefortheindex.
iftheindexisnon-sparse,thedocumentsmissingtheattributewillbecontainedintheindexwithakeyvalueofnull.
Hashindexessupportindexingarrayvaluesiftheindexattributenameisextendedwitha[*].
SkiplistIndex
Askiplistisasortedindexstructure.Itcanbeusedtoquicklyfinddocumentswithspecificattributevalues,forrangequeriesandforreturningdocumentsfromtheindexinsortedorder.SkiplistswillbeusedfromwithinAQLandseveralqueryfunctions,e.g.byExample,firstExampleetc.
Skiplistindexeswillbeusedforlookups,rangequeriesandsortingonlyifeitherallindexattributesareprovidedinaquery,orifaleftmostprefixoftheindexattributesisspecified.
Forexample,ifaskiplistindexiscreatedonattributesvalue1andvalue2,thefollowingfilterconditionscanusetheindex(note:the<=and>=operatorsareintentionallyomittedhereforthesakeofbrevity):
FILTERdoc.value1==...
FILTERdoc.value1<...
FILTERdoc.value1>...
FILTERdoc.value1>...&&doc.value1<...
FILTERdoc.value1==...&&doc.value2==...
FILTERdoc.value1==...&&doc.value2>...
FILTERdoc.value1==...&&doc.value2>...&&doc.value2<...
IndexBasics
87
Inordertouseaskiplistindexforsorting,theindexattributesmustbespecifiedintheSORTclauseofthequeryinthesameorderastheyappearintheindexdefinition.Skiplistindexesarealwayscreatedinascendingorder,buttheycanbeusedtoaccesstheindexedelementsinbothascendingordescendingorder.However,foracombinedindex(anindexonmultipleattributes)thisrequiresthatthesortordersinasinglequeryasspecifiedintheSORTclausemustbeeitherallascending(optionallyommittedasascendingisthedefault)oralldescending.
Forexample,iftheskiplistindexiscreatedonattributesvalue1andvalue2(inthisorder),thenthefollowingsortsclausescanusetheindexforsorting:
SORTvalue1ASC,value2ASC(anditsequivalentSORTvalue1,value2)SORTvalue1DESC,value2DESC
SORTvalue1ASC(anditsequivalentSORTvalue1)SORTvalue1DESC
Thefollowingsortclausescannotmakeuseoftheindexorder,andrequireanextrasortstep:
SORTvalue1ASC,value2DESC
SORTvalue1DESC,value2ASC
SORTvalue2(anditsequivalentSORTvalue2ASC)SORTvalue2DESC(becausefirstindexedattributevalue1isnotusedinsortclause)
Note:thelattertwosortclausescannotusetheindexbecausethesortclausedoesnotrefertoaleftmostprefixoftheindexattributes.
Skiplistscanoptionallybedeclaredunique,disallowingsavingthesamevalueintheindexedattribute.Theycanbesparseornon-sparse.
Thedifferenttypesofskiplistindexeshavethefollowingcharacteristics:
uniqueskiplistindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.Tryingtoinsertadocumentwiththesamekeyvalueasanalreadyexistingdocumentwillleadtoauniqueconstraintviolation.
Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueofnullintheindexattribute(s)willstillbeindexed.Akeyvalueofnullmayonlyoccuronceintheindex,sothistypeofindexcannotbeusedforoptionalattributes.
unique,sparseskiplistindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.Documentsinwhichatleastoneoftheindexattributesisnotsetorhasavalueofnullarenotincludedintheindex.Thistypeofindexcanbeusedtoensurethattherearenoduplicatekeysinthecollectionfordocumentswhichhavetheindexedattributesset.Astheindexwillexcludedocumentsforwhichtheindexedattributesarenullornotset,itcanbeusedforoptionalattributes.
non-uniqueskiplistindex:alldocumentsinthecollectionwillbeindexed.Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueofnullintheindexattribute(s)willstillbeindexed.Duplicatekeyvaluescanoccuranddonotleadtouniqueconstraintviolations.
non-unique,sparseskiplistindex:onlythosedocumentswillbeindexedthathavealltheindexedattributessettoavalueotherthannull.Itcanbeusedforoptionalattributes.
Theoperationalamortizedcomplexityforskiplistindexesislogarithmicallycorrelatedwiththenumberofdocumentsintheindex.
Skiplistindexessupportindexingarrayvaluesiftheindexattributenameisextendedwitha[*]`.
PersistentIndex
Thepersistentindexisasortedindexwithpersistence.Theindexentriesarewrittentodiskwhendocumentsarestoredorupdated.Thatmeanstheindexentriesdonotneedtoberebuiltfromthecollectiondatawhentheserverisrestartedortheindexedcollectionisinitiallyloaded.Thususingpersistentindexesmayreducecollectionloadingtimes.
Thepersistentindextypecanbeusedforsecondaryindexesatthemoment.Thatmeansthepersistentindexcurrentlycannotbemadetheonlyindexforacollection,becausetherewillalwaysbethein-memoryprimaryindexforthecollectioninaddition,andpotentiallymoreindexes(suchastheedgesindexforanedgecollection).
TheindeximplementationisusingtheRocksDBengine,anditprovideslogarithmiccomplexityforinsert,update,andremoveoperations.Asthepersistentindexisnotanin-memoryindex,itdoesnotstorepointersintotheprimaryindexasallthein-memoryindexesdo,butinsteaditstoresadocument'sprimarykey.Toretrieveadocumentviaapersistentindexviaanindexvaluelookup,therewillthereforebe
IndexBasics
88
anadditionalO(1)lookupintotheprimaryindextofetchtheactualdocument.
Asthepersistentindexissorted,itcanbeusedforpointlookups,rangequeriesandsortingoperations,butonlyifeitherallindexattributesareprovidedinaquery,orifaleftmostprefixoftheindexattributesisspecified.
GeoIndex
Userscancreateadditionalgeoindexesononeormultipleattributesincollections.Ageoindexisusedtofindplacesonthesurfaceoftheearthfast.
Thegeoindexstorestwo-dimensionalcoordinates.Itcanbecreatedoneithertwoseparatedocumentattributes(latitudeandlongitude)orasinglearrayattributethatcontainsbothlatitudeandlongitude.Latitudeandlongitudemustbenumericvalues.
Thegeoindexprovidesoperationstofinddocumentswithcoordinatesnearesttoagivencomparisoncoordinate,andtofinddocumentswithcoordinatesthatarewithinaspecifiableradiusaroundacomparisoncoordinate.
ThegeoindexisusedviadedicatedfunctionsinAQL,thesimplequeriesfunctionsanditisimplicitlyappliedwheninAQLaSORTorFILTERisusedwiththedistancefunction.Otherwiseitwillnotbeusedforothertypesofqueriesorconditions.
FulltextIndex
Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.Afulltextindexcanbecreatedonasingleattributeonly,andwillindexallwordscontainedindocumentsthathaveatextualvalueinthatattribute.Onlywordswitha(specifiable)minimumlengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedbylibicu,whichistakingintoaccounttheselectedlanguageprovidedatserverstart.Wordsareindexedintheirlower-casedform.Theindexsupportscompletematchqueries(fullwords)andprefixqueries,plusbasiclogicaloperationssuchasand,orandnotforcombiningpartialresults.
Thefulltextindexissparse,meaningitwillonlyindexdocumentsforwhichtheindexattributeissetandcontainsastringvalue.Additionally,onlywordswithaconfigurableminimumlengthwillbeincludedintheindex.
ThefulltextindexisusedviadedicatedfunctionsinAQLorthesimplequeries,butwillnotbeenabledforothertypesofqueriesorconditions.
Indexingattributesandsub-attributes
Top-levelaswellasnestedattributescanbeindexed.Forattributesatthetoplevel,theattributenamesalonearerequired.Toindexasinglefield,passanarraywithasingleelement(stringoftheattributekey)tothefieldsparameteroftheensureIndex()method.Tocreateacombinedindexovermultiplefields,simplyaddmorememberstothefieldsarray:
//{name:"Smith",age:35}
db.posts.ensureIndex({type:"hash",fields:["name"]})
db.posts.ensureIndex({type:"hash",fields:["name","age"]})
Toindexsub-attributes,specifytheattributepathusingthedotnotation:
//{name:{last:"Smith",first:"John"}}
db.posts.ensureIndex({type:"hash",fields:["name.last"]})
db.posts.ensureIndex({type:"hash",fields:["name.last","name.first"]})
Indexingarrayvalues
Ifanindexattributecontainsanarray,ArangoDBwillstoretheentirearrayastheindexvaluebydefault.Accessingindividualmembersofthearrayviatheindexisnotpossiblethisway.
Tomakeanindexinserttheindividualarraymembersintotheindexinsteadoftheentirearrayvalue,aspecialarrayindexneedstobecreatedfortheattribute.Arrayindexescanbesetuplikeregularhashorskiplistindexesusingthecollection.ensureIndex()function.Tomakeahashorskiplistindexanarrayindex,theindexattributenameneedstobeextendedwith[*]whencreatingtheindexandwhenfilteringinanAQLqueryusingtheINoperator.
Thefollowingexamplecreatesanarrayhashindexonthetagsattributeinacollectionnamedposts:
IndexBasics
89
db.posts.ensureIndex({type:"hash",fields:["tags[*]"]});
db.posts.insert({tags:["foobar","baz","quux"]});
ThisarrayindexcanthenbeusedforlookingupindividualtagsvaluesfromAQLqueriesviatheINoperator:
FORdocINposts
FILTER'foobar'INdoc.tags
RETURNdoc
Itispossibletoaddthearrayexpansionoperator[*],butitisnotmandatory.Youmayuseittoindicatethatanarrayindexisused,itispurelycosmetichowever:
FORdocINposts
FILTER'foobar'INdoc.tags[*]
RETURNdoc
ThefollowingFILTERconditionswillnotusethearrayindex:
FILTERdoc.tagsANY=='foobar'
FILTERdoc.tagsANYIN'foobar'
FILTERdoc.tagsIN'foobar'
FILTERdoc.tags=='foobar'
FILTER'foobar'==doc.tags
Itisalsopossibletocreateanindexonsubattributesofarrayvalues.Thismakessenseiftheindexattributeisanarrayofobjects,e.g.
db.posts.ensureIndex({type:"hash",fields:["tags[*].name"]});
db.posts.insert({tags:[{name:"foobar"},{name:"baz"},{name:"quux"}]});
Thefollowingquerywillthenusethearrayindex(thisdoesrequirethearrayexpansionoperator):
FORdocINposts
FILTER'foobar'INdoc.tags[*].name
RETURNdoc
Ifyoustoreadocumenthavingthearraywhichdoescontainelementsnothavingthesubattributesthisdocumentwillalsobeindexedwiththevaluenull,whichinArangoDBisequaltoattributenotexisting.
ArangoDBsupportscreatingarrayindexeswithasingle[*]operatorperindexattribute.Forexample,creatinganindexasfollowsisnotsupported:
db.posts.ensureIndex({type:"hash",fields:["tags[*].name[*].value"]});
Arrayvalueswillautomaticallybede-duplicatedbeforebeinginsertedintoanarrayindex.Forexample,ifthefollowingdocumentisinsertedintothecollection,theduplicatearrayvaluebarwillbeinsertedonlyonce:
db.posts.insert({tags:["foobar","bar","bar"]});
Thisisdonetoavoidredudantstorageofthesameindexvalueforthesamedocument,whichwouldnotprovideanybenefit.
Ifanarrayindexisdeclaredunique,thede-duplicationofarrayvalueswillhappenbeforeinsertingthevaluesintotheindex,sotheaboveinsertoperationwithtwoidenticalvaluesbarwillnotnecessarilyfail
Itwillalwaysfailiftheindexalreadycontainsaninstanceofthebarvalue.However,ifthevaluebarisnotalreadypresentintheindex,thenthede-duplicationofthearrayvalueswilleffectivelyleadtobarbeinginsertedonlyonce.
Toturnoffthededuplicationofarrayvalues,itispossibletosetthededuplicateattributeonthearrayindextofalse.Thedefaultvaluefordeduplicateistruehowever,sode-duplicationwilltakeplaceifnotexplicitlyturnedoff.
db.posts.ensureIndex({type:"hash",fields:["tags[*]"],deduplicate:false});
IndexBasics
90
//willfailnow
db.posts.insert({tags:["foobar","bar","bar"]});
Ifanarrayindexisdeclaredandyoustoredocumentsthatdonothaveanarrayatthespecifiedattributethisdocumentwillnotbeinsertedintheindex.Hencethefollowingobjectswillnotbeindexed:
db.posts.ensureIndex({type:"hash",fields:["tags[*]"]});
db.posts.insert({something:"else"});
db.posts.insert({tags:null});
db.posts.insert({tags:"thisisnoarray"});
db.posts.insert({tags:{content:[1,2,3]}});
Anarrayindexisabletoindexexplicitnullvalues.Whenqueriedfornullvalues,itwillonlyreturnthosedocumentshavingexplicitlynullstoredinthearray,itwillnotreturnanydocumentsthatdonothavethearrayatall.
db.posts.ensureIndex({type:"hash",fields:["tags[*]"]});
db.posts.insert({tags:null})//Willnotbeindexed
db.posts.insert({tags:[]})//Willnotbeindexed
db.posts.insert({tags:[null]});//Willbeindexedfornull
db.posts.insert({tags:[null,1,2]});//Willbeindexedfornull,1and2
Declaringanarrayindexassparsedoesnothaveaneffectonthearraypartoftheindex,thisinparticularmeansthatexplicitnullvaluesarealsoindexedinthesparseversion.Ifanindexiscombinedfromanarrayandanormalattributethesparsitywillapplyfortheattributee.g.:
db.posts.ensureIndex({type:"hash",fields:["tags[*]","name"],sparse:true});
db.posts.insert({tags:null,name:"alice"})//Willnotbeindexed
db.posts.insert({tags:[],name:"alice"})//Willnotbeindexed
db.posts.insert({tags:[1,2,3]})//Willnotbeindexed
db.posts.insert({tags:[1,2,3],name:null})//Willnotbeindexed
db.posts.insert({tags:[1,2,3],name:"alice"})
//Willbeindexedfor[1,"alice"],[2,"alice"],[3,"alice"]
db.posts.insert({tags:[null],name:"bob"})
//Willbeindexedfor[null,"bob"]
PleasenotethatfilteringusingarrayindexesonlyworksfromwithinAQLqueriesandonlyifthequeryfiltersontheindexedattributeusingtheINoperator.Theothercomparisonoperators(==,!=,>,>=,<,<=,ANY,ALL,NONE)currentlycannotusearrayindexes.
Vertexcentricindexes
Asmentionedabove,themostimportantindexesforgraphsaretheedgeindexes,indexingthe_fromand_toattributesofedgecollections.Theyprovideveryquickaccesstoalledgesoriginatinginorarrivingatagivenvertex,whichallowstoquicklyfindallneighboursofavertexinagraph.
Inmanycasesonewouldliketorunmorespecificqueries,forexamplefindingamongsttheedgesoriginatinginagivenvertexonlythosewiththe20latesttimestamps.Exactlythisisachievedwith"vertexcentricindexes".Inasensethesearelocalizedindexesforanedgecollection,whichsitateverysinglevertex.
Technically,theyareimplementedinArangoDBasindexes,whichsortthecompleteedgecollectionfirstby_fromandthenbyotherattributes.Ifweforexamplehaveaskiplistindexontheattributes_fromandtimestampofanedgecollection,wecananswertheabovequestionveryquicklywithasinglerangelookupintheindex.
SinceArangoDB3.0onecancreatesortedindexes(type"skiplist"and"persistent")thatindexthespecialedgeattributes_fromor_toandadditionallyotherattributes.SinceArangoDB3.1,theseareusedingraphtraversals,whenappropriateFILTERstatementsarefoundbytheoptimizer.
Forexample,tocreateavertexcentricindexoftheabovetype,youwouldsimplydo
db.edges.ensureIndex({"type":"skiplist","fields":["_from","timestamp"]});
Then,querieslike
IndexBasics
91
FORv,e,pIN1..1OUTBOUND"V/1"edges
FILTERe.timestampALL>="2016-11-09"
RETURNp
willbeconsiderablyfasterincasetherearemanyedgesoriginatinginvertex"V/1"butonlyfewwitharecenttimestamp.
IndexBasics
92
WhichIndextousewhen
ArangoDBautomaticallyindexesthe_keyattributeineachcollection.Thereisnoneedtoindexthisattributeseparately.Pleasenotethatadocument's_idattributeisderivedfromthe_keyattribute,andisthusimplicitlyindexed,too.
ArangoDBwillalsoautomaticallycreateanindexon_fromand_toinanyedgecollection,meaningincomingandoutgoingconnectionscanbedeterminedefficiently.
Indextypes
Userscandefineadditionalindexesononeormultipledocumentattributes.SeveraldifferentindextypesareprovidedbyArangoDB.Theseindexeshavedifferentusagescenarios:
hashindex:providesquickaccesstoindividualdocumentsif(andonlyif)allindexedattributesareprovidedinthesearchquery.Theindexwillonlybeusedforequalitycomparisons.Itdoesnotsupportrangequeriesandcannotbeusedforsorting.
Thehashindexisagoodcandidateifallormostqueriesontheindexedattribute(s)areequalitycomparisons.TheuniquehashindexprovidesanamortizedcomplexityofO(1)forinsert,update,removeandlookupoperations.Thenon-uniquehashindexprovidesO(1)inserts,updatesandremoves,andwillallowlookingupdocumentsbyindexvaluewithamortizedO(n)complexity,withnbeingthenumberofdocumentswiththatindexvalue.
Anon-uniquehashindexonanoptionaldocumentattributeshouldbedeclaredsparsesothatitwillnotindexdocumentsforwhichtheindexattributeisnotset.
skiplistindex:skiplistskeeptheindexedvaluesinanorder,sotheycanbeusedforequalitylookups,rangequeriesandforsorting.Forhighselectivityattributes,skiplistindexeswillhaveahigheroverheadthanhashindexes.Forlowselectivityattributes,skiplistindexeswillbemoreefficientthannon-uniquehashindexes.
Additionally,skiplistindexesallowmoreusecases(e.g.rangequeries,sorting)thanhashindexes.Furthermore,theycanbeusedforlookupsbasedonaleftmostprefixoftheindexattributes.
persistentindex:apersistentindexbehavesmuchlikethesortedskiplistindex,exceptthatallindexvaluesarepersistedondiskanddonotneedtoberebuiltinmemorywhentheserverisrestartedortheindexedcollectionisreloaded.Theoperationsinapersistentindexhavelogarithmiccomplexity,butoperationshavemayhaveahigherconstantfactorthantheoperationsinaskiplistindex,becausethepersistentindexmayneedtomakeextraroundtripstotheprimaryindextofetchtheactualdocuments.
Apersistentindexcanbeusedforequalitylookups,rangequeriesandforsorting.Forhighselectivityattributes,persistentindexeswillhaveahigheroverheadthanskiplistorhashindexes.
Persistentindexesallowmoreusecases(e.g.rangequeries,sorting)thanhashindexes.Furthermore,theycanbeusedforlookupsbasedonaleftmostprefixoftheindexattributes.Incontrasttothein-memoryskiplistindexes,persistentindexesdonotneedtoberebuiltin-memorysotheydon'tinfluencetheloadingtimeofcollectionsasotherin-memoryindexesdo.
geoindex:thegeoindexprovidedbyArangoDBallowssearchingfordocumentswithinaradiusaroundatwo-dimensionalearthcoordinate(point),ortofinddocumentswithareclosesttoapoint.Documentcoordinatescaneitherbespecifiedintwodifferentdocumentattributesorinasingleattribute,e.g.
{"latitude":50.9406645,"longitude":6.9599115}
or
{"coords":[50.9406645,6.9599115]}
GeoindexeswillbeinvokedviaspecialfunctionsorAQLoptimization.TheoptimizationcanbetriggeredwhenacollectionwithgeoindexisenumeratedandaSORTorFILTERstatementisusedinconjunctionwiththedistancefunction.
fulltextindex:afulltextindexcanbeusedtoindexallwordscontainedinaspecificattributeofalldocumentsinacollection.Onlywordswitha(specifiable)minimumlengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedbylibicu,whichistakingintoaccounttheselectedlanguageprovidedatserverstart.
Whichindextousewhen
93
Theindexsupportscompletematchqueries(fullwords)andprefixqueries.Fulltextindexeswillonlybeinvokedviaspecialfunctions.
Sparsevs.non-sparseindexes
Hashindexesandskiplistindexescanoptionallybecreatedsparse.Asparseindexdoesnotcontaindocumentsforwhichatleastoneoftheindexattributeisnotsetorcontainsavalueofnull.
Assuchdocumentsareexcludedfromsparseindexes,theymaycontainfewerdocumentsthantheirnon-sparsecounterparts.Thisenablesfasterindexingandcanleadtoreducedmemoryusageincasetheindexedattributedoesoccuronlyinsome,butnotalldocumentsofthecollection.Sparseindexeswillalsoreducethenumberofcollisionsinnon-uniquehashindexesincasenon-existingoroptionalattributesareindexed.
Inordertocreateasparseindex,anobjectwiththeattributesparsecanbeaddedtotheindexcreationcommands:
db.collection.ensureIndex({type:"hash",fields:["attributeName"],sparse:true});
db.collection.ensureIndex({type:"hash",fields:["attributeName1","attributeName2"],sparse:true});
db.collection.ensureIndex({type:"hash",fields:["attributeName"],unique:true,sparse:true});
db.collection.ensureIndex({type:"hash",fields:["attributeName1","attributeName2"],unique:true,sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName"],sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName1","attributeName2"],sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName"],unique:true,sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName1","attributeName2"],unique:true,sparse:true});
Whennotexplicitlyset,thesparseattributedefaultstofalsefornewindexes.Otherindexesthanhashandskiplistdonotsupportsparsity.
Assparseindexesmayexcludesomedocumentsfromthecollection,theycannotbeusedforalltypesofqueries.Sparsehashindexescannotbeusedtofinddocumentsforwhichatleastoneoftheindexedattributeshasavalueofnull.Forexample,thefollowingAQLquerycannotuseasparseindex,evenifonewascreatedonattributeattr:
FORdocIncollection
FILTERdoc.attr==null
RETURNdoc
Ifthelookupvalueisnon-constant,asparseindexmayormaynotbeused,dependingontheothertypesofconditionsinthequery.Iftheoptimizercansafelydeterminethatthelookupvaluecannotbenull,asparseindexmaybeused.Whenuncertain,theoptimizerwillnotmakeuseofasparseindexinaqueryinordertoproducecorrectresults.
Forexample,thefollowingqueriescannotuseasparseindexonattrbecausetheoptimizerwillnotknowbeforehandwhetherthevalueswhicharecomparedtodoc.attrwillincludenull:
FORdocIncollection
FILTERdoc.attr==SOME_FUNCTION(...)
RETURNdoc
FORotherINotherCollection
FORdocIncollection
FILTERdoc.attr==other.attr
RETURNdoc
Sparseskiplistindexescanbeusedforsortingiftheoptimizercansafelydetectthattheindexrangedoesnotincludenullforanyoftheindexattributes.
Notethatifyouintendtousejoinsitmaybeclevertousenon-sparsityandmaybeevenuniquenessforthatattribute,elseallitemscontainingthenullvaluewillmatchagainsteachotherandthusproducelargeresults.
Whichindextousewhen
94
IndexUtilizationInmostcasesArangoDBwilluseasingleindexpercollectioninagivenquery.AQLqueriescanusemorethanoneindexpercollectionwhenmultipleFILTERconditionsarecombinedwithalogicalORandthesecanbecoveredbyindexes.AQLquerieswilluseasingleindexpercollectionwhenFILTERconditionsarecombinedwithlogicalAND.
Creatingmultipleindexesondifferentattributesofthesamecollectionmaygivethequeryoptimizermorechoiceswhenpickinganindex.Creatingmultipleindexesondifferentattributescanalsohelpinspeedingupdifferentqueries,withFILTERconditionsondifferentattributes.
Itisoftenbeneficialtocreateanindexonmorethanjustoneattribute.Byaddingmoreattributestoanindex,anindexcanbecomemoreselectiveandthusreducethenumberofdocumentsthatqueriesneedtoprocess.
ArangoDB'sprimaryindexes,edgesindexesandhashindexeswillautomaticallyprovideselectivityestimates.Indexselectivityestimatesareprovidedinthewebinterface,thegetIndexes()returnvalueandintheexplain()outputforagivenquery.
Themoreselectiveanindexis,themoredocumentsitwillfilteronaverage.Theindexselectivityestimatesarethereforeusedbytheoptimizerwhencreatingqueryexecutionplanswhentherearemultipleindexestheoptimizercanchoosefrom.Theoptimizerwillthenselectacombinationofindexeswiththelowestestimatedtotalcost.Ingeneral,theoptimizerwillpicktheindexeswiththehighestestimatedselectivity.
Sparseindexesmayormaynotbepickedbytheoptimizerinaquery.Assparseindexesdonotcontainnullvalues,theywillnotbeusedforqueriesiftheoptimizercannotsafelydeterminewhetheraFILTERconditionincludesnullvaluesfortheindexattributes.Theoptimizerpolicyistoproducecorrectresults,regardlessofwhetherorwhichindexisusedtosatisfyFILTERconditions.Ifitisunsureaboutwhetherusinganindexwillviolatethepolicy,itwillnotmakeuseoftheindex.
Troubleshooting
WhenindoubtaboutwhetherandwhichindexeswillbeusedforexecutingagivenAQLquery,clicktheExplainbuttoninthewebinterfaceintheQueriesvieworusetheexplain()methodforthestatementasfollows(fromtheArangoShell):
varquery="FORdocINcollectionFILTERdoc.value>42RETURNdoc";
varstmt=db._createStatement(query);
stmt.explain();
Theexplain()commandwillreturnadetailedJSONrepresentationofthequery'sexecutionplan.TheJSONexplainoutputisintendedtobeusedbycode.Togetahuman-readableandmuchmorecompactexplanationofthequery,thereisanexplainertool:
varquery="FORdocINcollectionFILTERdoc.value>42RETURNdoc";
require("@arangodb/aql/explainer").explain(query);
Ifanyoftheexplainmethodsshowsthataqueryisnotusingindexes,thefollowingstepsmayhelp:
checkiftheattributenamesinthequeryarecorrectlyspelled.Inaschema-freedatabase,documentsinthesamecollectioncanhavevaryingstructures.Thereisnosuchthingasanon-existingattributeerror.Aquerythatreferstoattributenamesnotpresentinanyofthedocumentswillnotreturnanerror,andobviouslywillnotbenefitfromindexes.
checkthereturnvalueofthegetIndexes()methodforthecollectionsusedinthequeryandvalidatethatindexesareactuallypresentontheattributesusedinthequery'sfilterconditions.
ifindexesarepresentbutnotusedbythequery,theindexesmayhavethewrongtype.Forexample,ahashindexwillonlybeusedforequalitycomparisons(i.e.==)butnotforothercomparisontypessuchas<,<=,>,>=.Additionallyhashindexeswillonlybeusedifalloftheindexattributesareusedinthequery'sFILTERconditions.AskiplistindexwillonlybeusedifatleastitsfirstattributeisusedinaFILTERcondition.Ifadditionallyoftheskiplistindexattributesarespecifiedinthequery(fromleft-to-right),theymayalsobeusedandallowtofiltermoredocuments.
usingindexedattributesasfunctionparametersorinarbitraryexpressionswilllikelyleadtotheindexontheattributenotbeingused.Forexample,thefollowingquerieswillnotuseanindexonvalue:
IndexUtilization
95
FORdocINcollectionFILTERTO_NUMBER(doc.value)==42RETURNdoc
FORdocINcollectionFILTERdoc.value-1==42RETURNdoc
Inthesecasesthequeriesshouldberewrittensothatonlytheindexattributeispresentononesideoftheoperator,oradditionalfiltersandindexesshouldbeusedtorestricttheamountofdocumentsotherwise.
certainAQLfunctionssuchasWITHIN()orFULLTEXT()doutilizeindexesinternally,buttheiruseisnotmentionedinthequeryexplanationforfunctionsingeneral.Thesefunctionswillraisequeryerrors(atruntime)ifnosuitableindexispresentforthecollectioninquestion.
thequeryoptimizerwillingeneralpickoneindexpercollectioninaquery.ItcanpickmorethanoneindexpercollectioniftheFILTERconditioncontainsmultiplebranchescombinedwithlogicalOR.Forexample,thefollowingqueriescanuseindexes:
FORdocINcollectionFILTERdoc.value1==42||doc.value1==23RETURNdoc
FORdocINcollectionFILTERdoc.value1==42||doc.value2==23RETURNdoc
FORdocINcollectionFILTERdoc.value1<42||doc.value2>23RETURNdoc
ThetwoORsinthefirstquerywillbeconvertedtoanINlist,andifthereisasuitableindexonvalue1,itwillbeused.Thesecondqueryrequirestwoseparateindexesonvalue1andvalue2andwillusethemifpresent.Thethirdquerycanusetheindexesonvalue1andvalue2whentheyaresorted.
IndexUtilization
96
WorkingwithIndexes
IndexIdentifiersandHandles
Anindexhandleuniquelyidentifiesanindexinthedatabase.Itisastringandconsistsofthecollectionnameandanindexidentifierseparatedbya/.Theindexidentifierpartisanumericvaluethatisauto-generatedbyArangoDB.
Aspecificindexofacollectioncanbeaccessedusingitsindexhandleorindexidentifierasfollows:
db.collection.index("<index-handle>");
db.collection.index("<index-identifier>");
db._index("<index-handle>");
Forexample:Assumethattheindexhandle,whichisstoredinthe_idattributeoftheindex,isdemo/362549736andtheindexwascreatedinacollectionnameddemo.Thenthisindexcanbeaccessedas:
db.demo.index("demo/362549736");
Becausetheindexhandleisuniquewithinthedatabase,youcanleaveoutthecollectionandusetheshortcut:
db._index("demo/362549736");
CollectionMethods
Listingallindexesofacollection
returnsinformationabouttheindexesgetIndexes()
Returnsanarrayofallindexesdefinedforthecollection.
Notethat_keyimplicitlyhasanindexassignedtoit.
arangosh>db.test.ensureHashIndex("hashListAttribute",
........>"hashListSecondAttribute.subAttribute");
arangosh>db.test.getIndexes();
showexecutionresults
Creatinganindex
IndexesshouldbecreatedusingthegeneralmethodensureIndex.Thismethodobsoletesthespecializedindex-specificmethodsensureHashIndex,ensureSkiplist,ensureUniqueConstraintetc.
ensuresthatanindexexistscollection.ensureIndex(index-description)
Ensuresthatanindexaccordingtotheindex-descriptionexists.Anewindexwillbecreatedifnoneexistswiththegivendescription.
Theindex-descriptionmustcontainatleastatypeattribute.Otherattributesmaybenecessary,dependingontheindextype.
typecanbeoneofthefollowingvalues:
hash:hashindexskiplist:skiplistindexfulltext:fulltextindexgeo1:geoindex,withoneattributegeo2:geoindex,withtwoattributes
WorkingwithIndexes
97
sparsecanbetrueorfalse.
Forhash,andskiplistthesparsitycanbecontrolled,fulltextandgeoaresparsebydefinition.
uniquecanbetrueorfalseandissupportedbyhashorskiplist
Callingthismethodreturnsanindexobject.WhetherornottheindexobjectexistedbeforethecallisindicatedinthereturnattributeisNewlyCreated.
deduplicatecanbetrueorfalseandissupportedbyarrayindexesoftypehashorskiplist.Itcontrolswhetherinsertingduplicateindexvaluesfromthesamedocumentintoauniquearrayindexwillleadtoauniqueconstrainterrorornot.Thedefaultvalueistrue,soonlyasingleinstanceofeachnon-uniqueindexvaluewillbeinsertedintotheindexperdocument.Tryingtoinsertavalueintotheindexthatalreadyexistsintheindexwillalwaysfail,regardlessofthevalueofthisattribute.
Examples
arangosh>db.test.ensureIndex({type:"hash",fields:["a"],sparse:true});
arangosh>db.test.ensureIndex({type:"hash",fields:["a","b"],unique:true});
showexecutionresults
Droppinganindex
dropsanindexcollection.dropIndex(index)
Dropstheindex.Iftheindexdoesnotexist,thenfalseisreturned.Iftheindexexistedandwasdropped,thentrueisreturned.Notethatyoucannotdropsomespecialindexes(e.g.theprimaryindexofacollectionortheedgeindexofanedgecollection).
collection.dropIndex(index-handle)
Sameasabove.Insteadofanindexanindexhandlecanbegiven.
arangosh>db.example.ensureSkiplist("a","b");
arangosh>varindexInfo=db.example.getIndexes();
arangosh>indexInfo;
arangosh>db.example.dropIndex(indexInfo[0])
arangosh>db.example.dropIndex(indexInfo[1].id)
arangosh>indexInfo=db.example.getIndexes();
showexecutionresults
LoadIndexesintoMemory
LoadsallindexesofthiscollectionintoMemory.collection.loadIndexesIntoMemory()
Thisfunctiontriestocacheallindexentriesofthiscollectionintothemainmemory.Thereforeititeratesoverallindexesofthecollectionandstorestheindexedvalues,nottheentiredocumentdata,inmemory.Alllookupsthatcouldbefoundinthecachearemuchfasterthanlookupsnotstoredinthecachesoyougetaniceperformanceboost.Itisalsoguaranteedthatthecacheisconsistentwiththestoreddata.
ForthetimebeingthisfunctionisonlyusefulonRocksDBstorageengine,asinMMFilesengineallindexesareinmemoryanyways.
OnRocksDBthisfunctionhonorsallmemorylimits,iftheindexesyouwanttoloadaresmallerthanyourmemorylimitthisfunctionguaranteesthatmostindexvaluesarecached.Iftheindexislargerthanyourmemorylimitthisfunctionwillfillupvaluesuptothislimitandforthetimebeingthereisnowaytocontrolwhichindexesofthecollectionshouldhavepriorityoverothers.
arangosh>db.example.loadIndexesIntoMemory();
{
"result":true
}
WorkingwithIndexes
98
DatabaseMethods
Fetchinganindexbyhandle
findsanindexdb._index(index-handle)
Returnstheindexwithindex-handleornullifnosuchindexexists.
arangosh>db.example.ensureIndex({type:"skiplist",fields:["a","b"]});
arangosh>varindexInfo=db.example.getIndexes().map(function(x){returnx.id;});
arangosh>indexInfo;
arangosh>db._index(indexInfo[0])
arangosh>db._index(indexInfo[1])
showexecutionresults
Droppinganindex
dropsanindexdb._dropIndex(index)
Dropstheindex.Iftheindexdoesnotexist,thenfalseisreturned.Iftheindexexistedandwasdropped,thentrueisreturned.
db._dropIndex(index-handle)
Dropstheindexwithindex-handle.
arangosh>db.example.ensureIndex({type:"skiplist",fields:["a","b"]});
arangosh>varindexInfo=db.example.getIndexes();
arangosh>indexInfo;
arangosh>db._dropIndex(indexInfo[0])
arangosh>db._dropIndex(indexInfo[1].id)
arangosh>indexInfo=db.example.getIndexes();
showexecutionresults
Revalidatingwhetheranindexisused
findsanindex
Soyou'vecreatedanindex,andsinceitsmaintainanceisn'tforfree,youdefinitelywanttoknowwhetheryourquerycanutilizeit.
Youcanuseexplaintoverifywhetherskiplistsorhashindexesareused(ifyouomitcolors:falseyouwillgetnicecolorsinArangoShell):
arangosh>varexplain=require("@arangodb/aql/explainer").explain;
arangosh>db.example.ensureIndex({type:"skiplist",fields:["a","b"]});
arangosh>explain("FORdocINexampleFILTERdoc.a<23RETURNdoc",{colors:false});
showexecutionresults
WorkingwithIndexes
99
HashIndexes
IntroductiontoHashIndexes
Itispossibletodefineahashindexononeormoreattributes(orpaths)ofadocument.ThishashindexisthenusedinqueriestolocatedocumentsinO(1)operations.Ifthehashindexisunique,thennotwodocumentsareallowedtohavethesamesetofattributevalues.
Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheindexisdeclaredsparse,adocumentwillbeexcludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueofnull.
AccessingHashIndexesfromtheShell
UniqueHashIndexes
Ensuresthatauniqueconstraintexists:collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],unique:true})
Createsauniquehashindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.
Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocumentwillwillfailiftheattributeuniquenessisviolated.
Tocreateasparseuniqueindex,setthesparseattributetotrue:
collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],unique:true,sparse:true})
Incasethattheindexwassuccessfullycreated,theindexidentifierisreturned.
Non-existingattributeswilldefaulttonull.Inasparseindexalldocumentswillbeexcludedfromtheindexforwhichallspecifiedindexattributesarenull.Suchdocumentswillnotbetakenintoaccountforuniquenesschecks.
Inanon-sparseindex,alldocumentsregardlessofnull-attributeswillbeindexedandwillbetakenintoaccountforuniquenesschecks.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.test.ensureIndex({type:"hash",fields:["a","b.c"],unique:true});
arangosh>db.test.save({a:1,b:{c:1}});
arangosh>db.test.save({a:1,b:{c:1}});
arangosh>db.test.save({a:1,b:{c:null}});
arangosh>db.test.save({a:1});
showexecutionresults
Non-uniqueHashIndexes
Ensuresthatanon-uniquehashindexexists:collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"]})
Createsanon-uniquehashindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.
Tocreateasparseuniqueindex,setthesparseattributetotrue:
collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],sparse:true})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.test.ensureIndex({type:"hash",fields:["a"]});
arangosh>db.test.save({a:1});
HashIndexes
100
arangosh>db.test.save({a:1});
arangosh>db.test.save({a:null});
showexecutionresults
HashArrayIndexes
Ensuresthatahasharrayindexexists(non-unique):collection.ensureIndex({type:"hash",fields:["field1[*]",...,"fieldn[*]"]})
Createsanon-uniquehasharrayindexfortheindividualelementsofthearrayattributesfield1[*],...fieldn[*]foundinthedocuments.Atleastoneattributepathhastobegiven.Theindexalwaystreatstheindexedarraysassparse.
Itispossibletocombinearrayindexingwithstandardindexing:collection.ensureIndex({type:"hash",fields:["field1[*]","field2"]})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.test.ensureIndex({type:"hash",fields:["a[*]"]});
arangosh>db.test.save({a:[1,2]});
arangosh>db.test.save({a:[1,3]});
arangosh>db.test.save({a:null});
showexecutionresults
Ensureuniquenessofrelationsinedgecollections
Itispossibletocreatesecondaryindexesusingtheedgeattributes_fromand_to,startingwithArangoDB3.0.Acombinedindexoverbothfieldstogetherwiththeuniqueoptionenabledcanbeusedtopreventduplicaterelationsfrombeingcreated.
Forexample,adocumentcollectionvertsmightcontainverticeswiththedocumenthandlesverts/A,verts/Bandverts/C.Relationsbetweenthesedocumentscanbestoredinanedgecollectionedgesforinstance.Now,youmaywanttomakesurethatthevertexverts/Aisneverlinkedtoverts/Bbyanedgemorethanonce.Thiscanbeachievedbyaddingaunique,non-sparsehashindexforthefields_fromand_to:
db.edges.ensureIndex({type:"hash",fields:["_from","_to"],unique:true});
Creatinganedge{_from:"verts/A",_to:"verts/B"}inedgeswillbeaccepted,butonlyonce.AnotherattempttostoreanedgewiththerelationA→Bwillberejectedbytheserverwithauniqueconstraintviolatederror.Thisincludesupdatestothe_fromand_tofields.
NotethataddingarelationB→Aisstillpossible,soisA→AandB→B,becausetheyarealldifferentrelationsinadirectedgraph.Eachonecanonlyoccuroncehowever.
HashIndexes
101
Skiplists
IntroductiontoSkiplistIndexes
ThisisanintroductiontoArangoDB'sskiplists.
Itispossibletodefineaskiplistindexononeormoreattributes(orpaths)ofdocuments.Thisskiplististhenusedinqueriestolocatedocumentswithinagivenrange.Iftheskiplistisdeclaredunique,thennotwodocumentsareallowedtohavethesamesetofattributevalues.
Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheskiplistindexisdeclaredsparse,adocumentwillbeexcludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueofnull.
AccessingSkiplistIndexesfromtheShell
UniqueSkiplistIndex
Ensuresthatauniqueskiplistindexexists:collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],unique:true})
Createsauniqueskiplistindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.
Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocumentwillfailiftheattributeuniquenessisviolated.
Tocreateasparseuniqueindex,setthesparseattributetotrue:
collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],unique:true,sparse:true})
Inasparseindexalldocumentswillbeexcludedfromtheindexthatdonotcontainatleastoneofthespecifiedindexattributesorthathaveavalueofnullinanyofthespecifiedindexattributes.Suchdocumentswillnotbeindexed,andnotbetakenintoaccountforuniquenesschecks.
Inanon-sparseindex,thesedocumentswillbeindexed(fornon-presentindexedattributes,avalueofnullwillbeused)andwillbetakenintoaccountforuniquenesschecks.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.ids.ensureIndex({type:"skiplist",fields:["myId"],unique:true});
arangosh>db.ids.save({"myId":123});
arangosh>db.ids.save({"myId":456});
arangosh>db.ids.save({"myId":789});
arangosh>db.ids.save({"myId":123});
showexecutionresults
arangosh>db.ids.ensureIndex({type:"skiplist",fields:["name.first","name.last"],
unique:true});
arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});
arangosh>db.ids.save({"name":{"first":"jens","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});
showexecutionresults
Non-uniqueSkiplistIndex
Skiplists
102
Ensuresthatanon-uniqueskiplistindexexists:collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"]})
Createsanon-uniqueskiplistindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.
Tocreateasparsenon-uniqueindex,setthesparseattributetotrue.
collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],sparse:true})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.names.ensureIndex({type:"skiplist",fields:["first"]});
arangosh>db.names.save({"first":"Tim"});
arangosh>db.names.save({"first":"Tom"});
arangosh>db.names.save({"first":"John"});
arangosh>db.names.save({"first":"Tim"});
arangosh>db.names.save({"first":"Tom"});
showexecutionresults
SkiplistArrayIndex
Ensuresthataskiplistarrayindexexists(non-unique):collection.ensureIndex({type:"skiplist",fields:["field1[*]",...,"fieldn[*]"]})
Createsanon-uniqueskiplistarrayindexfortheindividualelementsofthearrayattributesfield1[*],...fieldn[*]foundinthedocuments.Atleastoneattributepathhastobegiven.Theindexalwaystreatstheindexedarraysassparse.
Itispossibletocombinearrayindexingwithstandardindexing:collection.ensureIndex({type:"skiplist",fields:["field1[*]","field2"]})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.test.ensureIndex({type:"skiplist",fields:["a[*]"]});
arangosh>db.test.save({a:[1,2]});
arangosh>db.test.save({a:[1,3]});
arangosh>db.test.save({a:null});
showexecutionresults
Querybyexampleusingaskiplistindex
Constructsaquery-by-exampleusingaskiplistindex:collection.byExample(example)
Selectsalldocumentsfromthecollectionthatmatchthespecifiedexampleandreturnsacursor.Askiplistindexwillbeusedifpresent.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.Ifyouuse
{"a":{"c":1}}
asexample,thenyouwillfindalldocuments,suchthattheattributeacontainsadocumentoftheform{c:1}.Forexamplethedocument
{"a":{"c":1},"b":1}
willmatch,butthedocument
{"a":{"c":1,"b":1}}
Skiplists
103
willnot.
However,ifyouuse
{"a.c":1},
thenyouwillfindalldocuments,whichcontainasub-documentinathathasanattributecofvalue1.Boththefollowingdocuments
{"a":{"c":1},"b":1}
and
{"a":{"c":1,"b":1}}
willmatch.
Skiplists
104
Persistentindexes
IntroductiontoPersistentIndexes
ThisisanintroductiontoArangoDB'spersistentindexes.
Itispossibletodefineapersistentindexononeormoreattributes(orpaths)ofdocuments.Theindexisthenusedinqueriestolocatedocumentswithinagivenrange.Iftheindexisdeclaredunique,thennotwodocumentsareallowedtohavethesamesetofattributevalues.
Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheindexisdeclaredsparse,adocumentwillbeexcludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueofnull.
AccessingPersistentIndexesfromtheShell
ensuresthatauniquepersistentindexexistscollection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"],unique:true})
Createsauniquepersistentindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.
Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocumentwillwillfailiftheattributeuniquenessisviolated.
Tocreateasparseuniqueindex,setthesparseattributetotrue:
collection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"],unique:true,sparse:true})
Inasparseindexalldocumentswillbeexcludedfromtheindexthatdonotcontainatleastoneofthespecifiedindexattributesorthathaveavalueofnullinanyofthespecifiedindexattributes.Suchdocumentswillnotbeindexed,andnotbetakenintoaccountforuniquenesschecks.
Inanon-sparseindex,thesedocumentswillbeindexed(fornon-presentindexedattributes,avalueofnullwillbeused)andwillbetakenintoaccountforuniquenesschecks.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.ids.ensureIndex({type:"persistent",fields:["myId"],unique:true});
arangosh>db.ids.save({"myId":123});
arangosh>db.ids.save({"myId":456});
arangosh>db.ids.save({"myId":789});
arangosh>db.ids.save({"myId":123});
showexecutionresults
arangosh>db.ids.ensureIndex({type:"persistent",fields:["name.first","name.last"],
unique:true});
arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});
arangosh>db.ids.save({"name":{"first":"jens","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});
showexecutionresultsensuresthatanon-uniquepersistentindexexistscollection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"]})
Createsanon-uniquepersistentindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.Theindexwillbenon-sparsebydefault.
Persistent
105
Tocreateasparseuniqueindex,setthesparseattributetotrue.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.names.ensureIndex({type:"persistent",fields:["first"]});
arangosh>db.names.save({"first":"Tim"});
arangosh>db.names.save({"first":"Tom"});
arangosh>db.names.save({"first":"John"});
arangosh>db.names.save({"first":"Tim"});
arangosh>db.names.save({"first":"Tom"});
showexecutionresults
Querybyexampleusingapersistentindex
constructsaquery-by-exampleusingapersistentindexcollection.byExample(example)
Selectsalldocumentsfromthecollectionthatmatchthespecifiedexampleandreturnsacursor.Apersistentindexwillbeusedifpresent.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.Ifyouuse
{"a":{"c":1}}
asexample,thenyouwillfindalldocuments,suchthattheattributeacontainsadocumentoftheform{c:1}.Forexamplethedocument
{"a":{"c":1},"b":1}
willmatch,butthedocument
{"a":{"c":1,"b":1}}
willnot.
However,ifyouuse
{"a.c":1},
thenyouwillfindalldocuments,whichcontainasub-documentinathathasanattributecofvalue1.Boththefollowingdocuments
{"a":{"c":1},"b":1}
and
{"a":{"c":1,"b":1}}
willmatch.
PersistentIndexesandServerLanguageTheorderofindexentriesinpersistentindexesadherestotheconfiguredserverlanguage.If,however,theserverisrestartedwithadifferentlanguagesettingaswhenthepersistentindexwascreated,notalldocumentsmaybereturnedanymoreandthesortorderofthosewhicharereturnedcanbewrong(wheneverthepersistentindexisconsulted).
Tofixpersistentindexesafteralanguagechange,deleteandre-createthem.Skiplistindexesarenotaffected,becausetheyarenotpersistedandautomaticallyrebuiltoneveryserverstart.
Persistent
106
Persistent
107
FulltextindexesThisisanintroductiontoArangoDB'sfulltextindexes.
IntroductiontoFulltextIndexes
Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.
Afulltextindexcanbedefinedononeattributeonly,andwillincludeallwordscontainedindocumentsthathaveatextualvalueintheindexattribute.SinceArangoDB2.6theindexwillalsoincludewordsfromtheindexattributeiftheindexattributeisanarrayofstrings,oranobjectwithstringvaluemembers.
Forexample,givenafulltextindexonthetranslationsattributeandthefollowingdocuments,thensearchingforлисаusingthefulltextindexwouldreturnonlythefirstdocument.SearchingfortheindexfortheexactstringFoxwouldreturnthefirsttwodocuments,andsearchingforprefix:Foxwouldreturnallthreedocuments:
{translations:{en:"fox",de:"Fuchs",fr:"renard",ru:"лиса"}}
{translations:"FoxistheEnglishtranslationoftheGermanwordFuchs"}
{translations:["ArangoDB","document","database","Foxx"]}
Notethatdeepernestedobjectsareignored.Forexample,afulltextindexontranslationswouldindexFuchs,butnotfox,giventhefollowingdocumentstructure:
{translations:{en:{US:"fox"},de:"Fuchs"}
Ifyouneedtosearchacrossmultiplefieldsand/ornestedobjects,youmaywriteallthestringsintoaspecialattribute,whichyouthencreatetheindexon(itmightbenecessarytocleanthestringsfirst,e.g.removelinebreaksandstripcertainwords).
Iftheindexattributeisneitherastring,anobjectoranarray,itscontentswillnotbeindexed.Whenindexingthecontentsofanarrayattribute,anarraymemberwillonlybeincludedintheindexifitisastring.Whenindexingthecontentsofanobjectattribute,anobjectmembervaluewillonlybeincludedintheindexifitisastring.Otherdatatypesareignoredandnotindexed.
Currently,fulltextindexesarenotyetsupportedwiththeRocksDBstorageengine.ThusthefunctionFULLTEXT()willbeunavailablewhenusingthisstorageengine.Tousefulltextindexes,pleaseusetheMMFilesstorageengineforthetimebeing.
AccessingFulltextIndexesfromtheShell
Ensuresthatafulltextindexexists:
collection.ensureIndex({type:"fulltext",fields:["field"],minLength:minLength})
Createsafulltextindexonalldocumentsonattributefield.
Fulltextindexesareimplicitlysparse:alldocumentswhichdonothavethespecifiedfieldattributeorthathaveanon-qualifyingvalueintheirfieldattributewillbeignoredforindexing.
Onlyasingleattributecanbeindexed.Specifyingmultipleattributesisunsupported.
TheminimumlengthofwordsthatareindexedcanbespecifiedviatheminLengthparameter.WordsshorterthanminLengthcharacterswillnotbeindexed.minLengthhasadefaultvalueof2,butthisvaluemightbechangedinfutureversionsofArangoDB.Itisthusrecommendedtoexplicitlyspecifythisvalue.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetailsisreturned.
arangosh>db.example.ensureIndex({type:"fulltext",fields:["text"],minLength:3});
arangosh>db.example.save({text:"thequickbrown",b:{c:1}});
arangosh>db.example.save({text:"quickbrownfox",b:{c:2}});
arangosh>db.example.save({text:"brownfoxjums",b:{c:3}});
arangosh>db.example.save({text:"foxjumpsover",b:{c:4}});
FulltextIndexes
108
arangosh>db.example.save({text:"jumpsoverthe",b:{c:5}});
arangosh>db.example.save({text:"overthelazy",b:{c:6}});
arangosh>db.example.save({text:"thelazydog",b:{c:7}});
arangosh>db._query("FORdocumentINFULLTEXT(example,'text','the')RETURNdocument");
showexecutionresultsLooksupafulltextindex:
collection.lookupFulltextIndex(attribute,minLength)
Checkswhetherafulltextindexonthegivenattributeattributeexists.
FulltextAQLFunctions
FulltextAQLfunctionsaredetailedinFulltextfunctions.
FulltextIndexes
109
GeoIndexes
IntroductiontoGeoIndexes
ThisisanintroductiontoArangoDB'sgeoindexes.
AQL'sgeographicfeaturesaredescribedinGeofunctions.
ArangoDBusesHilbertcurvestoimplementgeo-spatialindexes.Seethisblogfordetails.
Ageo-spatialindexassumesthatthelatitudeisbetween-90and90degreeandthelongitudeisbetween-180and180degree.Ageoindexwillignorealldocumentswhichdonotfulfilltheserequirements.
AccessingGeoIndexesfromtheShell
ensuresthatageoindexexistscollection.ensureIndex({type:"geo",fields:["location"]})
Createsageo-spatialindexonalldocumentsusinglocationaspathtothecoordinates.Thevalueoftheattributehastobeanarraywithatleasttwonumericvalues.Thearraymustcontainthelatitude(firstvalue)andthelongitude(secondvalue).
Alldocuments,whichdonothavetheattributepathorhaveanon-conformingvalueinitareexcludedfromtheindex.
Ageoindexisimplicitlysparse,andthereisnowaytocontrolitssparsity.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
Tocreateageoindexonanarrayattributethatcontainslongitudefirst,setthegeoJsonattributetotrue.ThiscorrespondstotheformatdescribedinRFC7946Position
collection.ensureIndex({type:"geo",fields:["location"],geoJson:true})
Tocreateageo-spatialindexonalldocumentsusinglatitudeandlongitudeasseparateattributepaths,twopathsneedtobespecifiedinthefieldsarray:
collection.ensureIndex({type:"geo",fields:["latitude","longitude"]})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
Examples
Createageoindexforanarrayattribute:
arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});
........>}
........>}
arangosh>db.geo.count();
arangosh>db.geo.near(0,0).limit(3).toArray();
arangosh>db.geo.near(0,0).count();
showexecutionresultsCreateageoindexforahasharrayattribute:
arangosh>db.geo2.ensureIndex({type:"geo",fields:["location.latitude",
"location.longitude"]});
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.geo2.save({name:"Name/"+i+"/"+j,location:{latitude:i,
longitude:j}});
GeoIndexes
110
........>}
........>}
arangosh>db.geo2.near(0,0).limit(3).toArray();
showexecutionresultsUseGeoIndexwithAQLSORTstatement:
arangosh>db.geoSort.ensureIndex({type:"geo",fields:["latitude","longitude"]});
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.geoSort.save({name:"Name/"+i+"/"+j,latitude:i,longitude
:j});
........>}
........>}
arangosh>varquery="FORdocingeoSortSORTDISTANCE(doc.latitude,doc.longitude,0,0)
LIMIT5RETURNdoc"
arangosh>db._explain(query,{},{colors:false});
arangosh>db._query(query);
showexecutionresultsUseGeoIndexwithAQLFILTERstatement:
arangosh>db.geoFilter.ensureIndex({type:"geo",fields:["latitude","longitude"]});
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.geoFilter.save({name:"Name/"+i+"/"+j,latitude:i,
longitude:j});
........>}
........>}
arangosh>varquery="FORdocingeoFilterFILTERDISTANCE(doc.latitude,doc.longitude,
0,0)<2000RETURNdoc"
arangosh>db._explain(query,{},{colors:false});
arangosh>db._query(query);
showexecutionresultsconstructsageoindexselectioncollection.geo(location-attribute)Looksupageoindexdefinedonattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Thisisusefulforcollectionswithmultipledefinedgeoindexes.collection.geo(location_attribute,true)Looksupageoindexonacompoundattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.collection.geo(latitude_attribute,longitude_attribute)Looksupageoindexdefinedonthetwoattributeslatitude_attributeandlongitude-attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thegeosimplequeryhelperfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforrunninggeoqueriesistousetheirAQLequivalents.
Examples
Assumeyouhavealocationstoredaslistintheattributehomeandadestinationstoredintheattributework.Thenyoucanusethegeooperatortoselectwhichgeo-spatialattributes(andthuswhichindex)touseinanearquery.
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.complex.save({name:"Name/"+i+"/"+j,
........>home:[i,j],
........>work:[-i,-j]});
........>}
GeoIndexes
111
........>}
........>
arangosh>db.complex.near(0,170).limit(5);
arangosh>db.complex.ensureIndex({type:"geo",fields:["home"]});
arangosh>db.complex.near(0,170).limit(5).toArray();
arangosh>db.complex.geo("work").near(0,170).limit(5);
arangosh>db.complex.ensureIndex({type:"geo",fields:["work"]});
arangosh>db.complex.geo("work").near(0,170).limit(5).toArray();
showexecutionresultsconstructsanearqueryforacollectioncollection.near(latitude,longitude)Thereturnedlistissortedaccordingtothedistance,withthenearestdocumenttothecoordinate(latitude,longitude)comingfirst.Ifthereareneardocumentsofequaldistance,documentsarechosenrandomlyfromthissetuntilthelimitisreached.Itispossibletochangethelimitusingthelimitoperator.Inordertousethenearoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesforthedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.Note:neardoesnotsupportnegativeskips.//However,youcanstilluselimitfollowedtoskip.collection.near(latitude,longitude).limit(limit)Limitstheresulttolimitdocumentsinsteadofthedefault100.Note:Unlikewithmultipleexplicitlimits,limitwillraisetheimplicitdefaultlimitimposedbywithin.collection.near(latitude,longitude).distance()Thiswilladdanattributedistancetoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.collection.near(latitude,longitude).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thenearsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionusingthenearoperatoristousetheAQLNEARfunctioninanAQLqueryasfollows:
FORdocINNEAR(@@collection,@latitude,@longitude,@limit)
RETURNdoc
Examples
Togetthenearesttwolocations:
arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});
arangosh>for(vari=-90;i<=90;i+=10){
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({
........>name:"Name/"+i+"/"+j,
........>loc:[i,j]});
........>}}
arangosh>db.geo.near(0,0).limit(2).toArray();
showexecutionresultsIfyouneedthedistanceaswell,thenyoucanusethedistanceoperator:
arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});
arangosh>for(vari=-90;i<=90;i+=10){
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({
........>name:"Name/"+i+"/"+j,
........>loc:[i,j]});
........>}}
arangosh>db.geo.near(0,0).distance().limit(2).toArray();
showexecutionresultsconstructsawithinqueryforacollectioncollection.within(latitude,longitude,radius)Thiswillfindalldocumentswithinagivenradiusaroundthecoordinate(latitude,longitude).Thereturnedarrayissortedbydistance,beginningwiththenearestdocument.Inordertousethewithinoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesfor
GeoIndexes
112
thedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.collection.within(latitude,longitude,radius).distance()Thiswilladdanattribute_distancetoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.collection.within(latitude,longitude,radius).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thewithinsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionusingthewithinoperatoristousetheAQLWITHINfunctioninanAQLqueryasfollows:
FORdocINWITHIN(@@collection,@latitude,@longitude,@radius,@distanceAttributeName)
RETURNdoc
Examples
Tofindalldocumentswithinaradiusof2000kmuse:
arangosh>for(vari=-90;i<=90;i+=10){
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});}}
arangosh>db.geo.within(0,0,2000*1000).distance().toArray();
showexecutionresultsensuresthatageoindexexistscollection.ensureIndex({type:"geo",fields:["location"]})
SinceArangoDB2.5,thismethodisanaliasforensureGeoIndexsincegeoindexesarealwayssparse,meaningthatdocumentsthatdonotcontaintheindexattributesorhavenon-numericvaluesintheindexattributeswillnotbeindexed.ensureGeoConstraintisdeprecatedandensureGeoIndexshouldbeusedinstead.
Theindexdoesnotprovideauniqueoptionbecauseofitslimitedusability.Itwouldpreventidenticalcoordinatesfrombeinginsertedonly,butevenaslightlydifferentlocation(like1inchor1cmoff)wouldbeuniqueagainandnotconsideredaduplicate,althoughitprobablyshould.Thedesiredthresholdfordetectingduplicatesmayvaryforeveryproject(includinghowtocalculatethedistanceeven)andneedstobeimplementedontheapplicationlayerasneeded.YoucanwriteaFoxxserviceforthispurposeandmakeuseoftheAQLgeofunctionstofindnearbycoordinatessupportedbyageoindex.
GeoIndexes
113
VertexCentricIndexes
IntroductiontoVertexCentricIndexes
InArangoDBtherearespecialindicesdesignedtospeedupgraphoperations,especiallyifthegraphcontainssupernodes(verticesthathaveanexceptionallyhighamountofconnectededges).Theseindicesarecalledvertexcentricindexesandcanbeusedinadditiontotheexistingedgeindex.
Motivation
Theideaofthisindexistoindexacombinationofavertex,thedirectionandanyarbitrarysetofotherattributesontheedges.Totakeanexample,ifwehaveanattributecalledtypeontheedges,wecanuseanoutboundvertex-centricindexonthisattributetofindalledgesattachedtoavertexwithagiventype.Thefollowingqueryexamplecouldbenefitfromsuchanindex:
FORv,e,pIN3..5OUTBOUND@startGRAPH@graphName
FILTERp.edges[*].typeALL=="friend"
RETURNv
Usingthebuilt-inedge-indexArangoDBcanfindthelistofalledgesattachedtothevertexfast,butstillithastowalkthroughthislistandcheckifallofthemhavetheattributetype=="friend".Usingavertex-centricindexwouldallowArangoDBtofindalledgesforthevertexhavingtheattributetype=="friend"inthesametimeandcansavetheiterationtoverifythecondition.
IndexcreationAvertex-centriccanbeeitherofthefollowingtypes:
HashIndexSkiplistIndexPersistentIndex
Andiscreatedusingtheircreationoperations.Howeverinthelistoffieldsusedtocreatetheindexwehavetoincludeeither_fromor_to.Letusagainexplainthisbyanexample.Assumewewanttocreateanhash-basedoutboundvertex-centricindexontheattributetype.Thiscanbecreatedwiththefollowingway:
arangosh>db.collection.ensureIndex({type:"hash",fields:["_from","type"]})
showexecutionresultsAlloptionsthataresupportedbytherespectiveindexesaresupportedbythevertex-centricindexaswell.
Indexusage
TheAQLoptimizercandecidetouseavertex-centricwheneversuitable,howeveritisnotguaranteedthatthisindexisused,theoptimizermayestimatethatanotherindexisassumedtobebetter.Theoptimizerwillconsiderthistypeofindexesonexplicitfilteringof_fromrespectively_to:
FORedgeINcollection
FILTERedge._from=="vertices/123456"ANDedge.type=="friend"
RETURNedge
andduringpatternmatchingqueries:
FORv,e,pIN3..5OUTBOUND@startGRAPH@graphName
FILTERp.edges[*].typeALL=="friend"
RETURNv
VertexCentricIndexes
114
VertexCentricIndexes
115
ArangoDBGraphs
FirstStepswithGraphs
AGraphconsistsofverticesandedges.Edgesarestoredasdocumentsinedgecollections.Avertexcanbeadocumentofadocumentcollectionorofanedgecollection(soedgescanbeusedasvertices).Whichcollectionsareusedwithinanamedgraphisdefinedviaedgedefinitions.Anamedgraphcancontainmorethanoneedgedefinition,atleastoneisneeded.Graphsallowyoutostructureyourmodelsinlinewithyourdomainandgroupthemlogicallyincollectionsandgivingyouthepowertoquerytheminthesamegraphqueries.
Newtographs?Takeourfreegraphcourseforfreshersandgetfromzeroknowledgetoadvancedquerytechniques.
Comingfromarelationalbackground-what'sagraph?
InSQLyoucommonlyhavetheconstructofarelationtabletostoren:mrelationsbetweentwodatatables.Anedgecollectionissomewhatsimilartotheserelationtables;vertexcollectionsresemblethedatatableswiththeobjectstoconnect.WhilesimplegraphquerieswithfixednumberofhopsviatherelationtablemaybedoableinSQLwithseveralnestedjoins,graphdatabasescanhandleanarbitrarynumberofthesehopsoveredgecollections-thisiscalledtraversal.Alsoedgesinoneedgecollectionmaypointtoseveralvertexcollections.Itscommontohaveattributesattachedtoedges,i.e.alabelnamingthisinterconnection.Edgeshaveadirection,withtheirrelations_fromand_topointingfromonedocumenttoanotherdocumentstoredinvertexcollections.Inqueriesyoucandefineinwhichdirectionstheedgerelationsmaybefollowed(OUTBOUND:_from→_to,INBOUND:_from←_to,ANY:_from↔_to).
NamedGraphs
NamedgraphsarecompletelymanagedbyArangoDB,andthusalsovisibleinthewebinterface.TheyusethefullspectrumofArangoDB'sgraphfeatures.Youmayaccessthemviaseveralinterfaces.
AQLGraphOperationswithseveralflavors:AQLTraversalsonbothnamedandanonymousgraphsAQLShortestPathonbothnamedandanonymousgraph
JavaScriptGeneralGraphimplementation,asyoumayuseitinFoxxServicesGraphManagement;creating&manipualatinggraphdefinitions;inserting,updatinganddeletingverticesandedgesintographsGraphFunctionsforworkingwithedgesandvertices,toanalyzethemandtheirrelations
JavaScriptSmartGraphimplementation,forscalablegraphsSmartGraphManagement;creating&manipualatingSmartGraphdefinitions;DifferencestoGeneralGraph
RESTfulGeneralGraphinterfaceusedtoimplementgraphmanagementinclientdrivers
Manipulatingcollectionsofnamedgraphswithregulardocumentfunctions
Theunderlyingcollectionsofthenamedgraphsarestillaccessibleusingthestandardmethodsforcollections.Howeverthegraphmoduleaddsanadditionallayerontopofthesecollectionsgivingyouthefollowingguarantees:
AllmodificationsareexecutedtransactionalIfyoudeleteavertexalledgeswillbedeleted,youwillneverhavelooseendsIfyouinsertanedgeitischeckediftheedgematchestheedgedefinitions,youredgecollectionswillonlycontainvalidedges
TheseguaranteesarelostifyouaccessthecollectionsinanyotherwaythanthegraphmoduleorAQL,soifyoudeletedocumentsfromyourvertexcollectionsdirectly,theedgespointingtothemwillberemaininplace.
Anonymousgraphs
Sometimesyoumaynotneedallthepowersofnamedgraphs,butsomeofitsbitsmaybevaluabletoyou.YoumayuseanonymousgraphsinthetraversalsandintheWorkingwithEdgeschapter.Anonymousgraphsdon'thaveedgedefinitionsdescribingwhichvertexcollectionisconnectedbywhichedgecollection.Thegraphmodelhastobemaintainedintheclientsidecode.Thisgivesyoumorefreedomthanthestrictnamedgraphs.
Graphs
116
AQLGraphOperationsareavailableforboth,namedandanonymousgraphs:AQLTraversalsAQLShortestPath
Whentochooseanonymousornamedgraphs?
Asnotedabove,namedgraphsensuregraphintegrity,bothwheninsertingorremovingedgesorvertices.Soyouwon'tencounterdanglingedges,evenifyouusethesamevertexcollectioninseveralnamedgraphs.Thisinvolvesmoreoperationsinsidethedatabasewhichcomeatacost.Thereforeanonymousgraphsmaybefasterinmanyoperations.Sothisquestionmaybenarroweddownto:'CanIaffordtheadditionaleffortordoIneedthewarrantyforintegrity?'.
Multipleedgecollectionsvs.FILTERsonedgedocumentattributes
Ifyouwanttoonlytraverseedgesofaspecifictype,therearetwowaystoachievethis.Thefirstwouldbeanattributeintheedgedocument-i.e.type,whereyouspecifyadifferentiatorfortheedge-i.e."friends","family","married"or"workmates",soyoucanlaterFILTERe.type="friends"ifyouonlywanttofollowthefriendedges.
Anotherway,whichmaybemoreefficientinsomecases,istousedifferentedgecollectionsfordifferenttypesofedges,soyouhavefriend_edges,family_edges,married_edgesandworkmate_edgesascollectionnames.Youcanthenconfigureseveralnamedgraphsincludingasubsetoftheavailableedgeandvertexcollections-oryouuseanonymousgraphqueries,whereyouspecifyalistofedgecollectionstotakeintoaccountinthatquery.Toonlyfollowfriendedges,youwouldspecifyfriend_edgesassoleedgecollection.
Bothapproacheshaveadvantagesanddisadvantages.FILTERoperationsonedeattributeswilldocomparisonsoneachtraversededge,whichmaybecomeCPU-intense.Whennotfindingtheedgesinthefirstplacebecauseofthecollectioncontainingthemisnottraversedatall,therewillneverbeareasontoactualycheckfortheirtypeattributewithFILTER.
Themultipleedgecollectionsapproachislimitedbythenumberofcollectionsthatcanbeusedsimultaneouslyinonequery.EverycollectionusedinaqueryrequiressomeresourcesinsideofArangoDBandthenumberisthereforelimitedtocaptheresourcerequirements.Youmayalsohaveconstraintsonotheredgeattributes,suchasahashindexwithauniqueconstraint,whichrequiresthedocumentstobeinasinglecollectionfortheuniquenessguarantee,anditmaythusnotbepossibletostorethedifferenttypesofedgesinmultipleedegcollections.
So,ifyouredgeshaveaboutadozendifferenttypes,it'sokaytochoosethecollectionapproach,otherwisetheFILTERapproachispreferred.YoucanstilluseFILTERoperationsonedgesofcourse.YoucangetridofaFILTERonthetypewiththeformerapproach,everythingelsecanstaythesame.
WhichpartofmydataisanEdgeandwhichaVertex?
Themainobjectsinyourdatamodel,suchasusers,groupsorarticles,areusuallyconsideredtobevertices.Foreachtypeofobject,adocumentcollection(alsocalledvertexcollection)shouldstoretheindividualentities.Entitiescanbeconnectedbyedgestoexpressandclassifyrelationsbetweenvertices.Itoftenmakessensetohaveanedgecollectionperrelationtype.
ArangoDBdoesnotrequireyoutostoreyourdataingraphstructureswithedgesandvertices,youcanalsodecidetoembedattributessuchaswhichgroupsauserispartof,or_idsofdocumentsinanotherdocumentinsteadofconnectingthedocumentswithedges.Itcanbeameaningfulperformanceoptimizationfor1:nrelationships,ifyourdataisnotfocusedonrelationsandyoudon'tneedgraphtraversalwithvaryingdepth.Itusuallymeanstointroducesomeredundancyandpossiblyinconsistenciesifyouembeddata,butitcanbeanacceptabletradeoff.
Vertices
Let'ssaywehavetwovertexcollections,UsersandGroups.DocumentsintheGroupscollectioncontaintheattributesoftheGroup,i.e.whenitwasfounded,itssubject,aniconURLandsoon.Usersdocumentscontainthedataspecifictoauser-likeallnames,birthdays,AvatarURLs,hobbies...
Edges
Wecanuseanedgecollectiontostorerelationsbetweenusersandgroups.Sincemultipleusersmaybeinanarbitrarynumberofgroups,thisisanm:nrelation.TheedgecollectioncanbecalledUsersInGroupswithi.e.oneedgewith_frompointingtoUsers/Johnand_topointingtoGroups/BowlingGroupHappyPin.ThismakestheuserJohnamemberofthegroupBowlingGroupHappyPin.
Graphs
117
Attributesofthisrelationmaycontainqualifierstothisrelation,likethepermissionsofJohninthisgroup,thedatewhenhejoinedthegroupetc.
Soroughlyput,ifyouusedocumentsandtheirattributesinasentence,nounswouldtypicallybevertices,verbsbecometheedges.Youcanseethisintheknowsgraphbelow:
AliceknowsBob,whointermknowsCharlie.
Advantagesofthisapproach
Graphsgiveyoutheadvantageofnotjustbeingabletohaveafixednumberofm:nrelationsinarow,butanarbitrarynumber.Edgescanbetraversedinbothdirections,soit'seasytodetermineallgroupsauserisin,butalsotofindoutwhichmembersacertaingrouphas.Userscouldalsobeinterconnectedtocreateasocialnetwork.
Usingthegraphdatamodel,dealingwithdatathathaslotsofrelationsstaysmanageableandcanbequeriedinveryflexibleways,whereasitwouldcauseheadachetohandleitinarelationaldatabasesystem.
Backupandrestore
Forsureyouwanttohavebackupsofyourgraphdata,youcanuseArangodumptocreatethebackup,andArangorestoretorestoreabackupintoanewArangoDB.Youshouldhowevernotethat:
youneedthesystemcollection_graphsifyoubackupnamedgraphs.youneedtobackupthecompletesetofalledgeandvertexcollectionsyourgraphconsistsof.Partialdump/restoremaynotwork.
Managinggraphs
Bydefaultyoushouldusetheinterfaceyourdriverprovidestomanagegraphs.
Thisisi.e.documentedinGraphs-SectionoftheArangoDBJavadriver.
ExampleGraphs
ArangoDBcomeswithasetofeasilygraspablegraphsthatareusedtodemonstratetheAPIs.Youcanusetheaddsamplestabinthecreategraphwindowinthewebinterface,orloadthemodule@arangodb/graph-examples/example-graphinarangoshanduseittocreateinstancesofthesegraphsinyourArangoDB.Onceyou'vecreatedthem,youcaninspecttheminthewebinterface-whichwasusedtocreatethepicturesbelow.
Youcaneasilylookintotheinnardsofthisscriptforreferenceabouthowtomanagegraphsprogramatically.
TheKnows_Graph
Graphs
118
Asetofpersonsknowingeachother:
Theknowsgraphconsistsofonevertexcollectionpersonsconnectedviaoneedgecollectionknows.ItwillcontainfivepersonsAlice,Bob,Charlie,DaveandEve.Wewillhavethefollowingdirectedrelations:
AliceknowsBobBobknowsCharlieBobknowsDaveEveknowsAliceEveknowsBob
Thisishowwecreateit,inspectitsverticesandedges,anddropitagain:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("knows_graph");
arangosh>db.persons.toArray()
arangosh>db.knows.toArray();
arangosh>examples.dropGraph("knows_graph");
showexecutionresults
TheSocialGraph
Asetofpersonsandtheirrelations:
Graphs
119
Thisexamplehasfemaleandmalepersonsasverticesintwovertexcollections-femaleandmale.Theedgesaretheirconnectionsintherelationedgecollection.Thisishowwecreateit,inspectitsverticesandedges,anddropitagain:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>db.female.toArray()
arangosh>db.male.toArray()
arangosh>db.relation.toArray()
arangosh>examples.dropGraph("social");
showexecutionresults
TheCityGraph
Asetofeuropeancities,andtheirfictionaltravelingdistancesasconnections:
Graphs
120
Theexamplehasthecitiesasverticesinseveralvertexcollections-germanCityandfrenchCity.Theedgesaretheirinterconnectionsinseveraledgecollectionsfrench/german/internationalHighway.Thisishowwecreateit,inspectitsedgesandvertices,anddropitagain:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>db.frenchCity.toArray();
arangosh>db.germanCity.toArray();
arangosh>db.germanHighway.toArray();
arangosh>db.frenchHighway.toArray();
arangosh>db.internationalHighway.toArray();
arangosh>examples.dropGraph("routeplanner");
showexecutionresults
TheTraversalGraph
Thisgraphwasdesignedtodemonstratefiltersintraversals.Ithassomelabelstofilteronit.
Graphs
121
Theexamplehasallitsverticesinthecirclescollection,andanedgesedgecollectiontoconnectthem.Circleshaveuniquenumericlabels.Edgeshavetwobooleanattributes(theFalsealwaysbeingfalse,theTruthalwaysbeingtrue)andalabelsortingB-Dtotheleftside,G-Ktotherightside.LeftandrightsidesplitintoPaths-atBandGwhichareeachdirectneighboursoftheroot-nodeA.StartingfromAthegraphhasadepthof3onallitspaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("traversalGraph");
arangosh>db.circles.toArray();
arangosh>db.edges.toArray();
arangosh>examples.dropGraph("traversalGraph");
showexecutionresults
TheWorldGraph
Theworldcountrygraphstructuresitsnodeslikethat:world→continent→country→capital.Insomecasesedgedirectionsaren'tforward(thereforeitwillbedisplayeddisjunctinthegraphviewer).Ithastwowaysofcreatingit.Oneusingthenamedgraphutilities(worldCountry),onewithout(worldCountryUnManaged).Itisusedtodemonstraterawtraversaloperations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("worldCountry");
arangosh>db.worldVertices.toArray();
arangosh>db.worldEdges.toArray();
arangosh>examples.dropGraph("worldCountry");
arangosh>varg=examples.loadGraph("worldCountryUnManaged");
arangosh>examples.dropGraph("worldCountryUnManaged");
showexecutionresults
Graphs
122
Cookbookexamples
TheabovereferencedchaptersdescribethevariousAPIsofArangoDBsgraphenginewithsmallexamples.Ourcookbookhassomemorereallifeexamples:
TraversingagraphinfulldepthUsinganexamplevertexwiththejavadriverRetrievingdocumentsfromArangoDBwithoutknowingthestructureUsingacustomvisitorfromnode.jsAQLExampleQueriesonanActorsandMoviesDatabase
Highervolumegraphexamples
Alloftheaboveexamplesarerathersmallsotheyareeasiertocomprehendandcandemonstratethewaythefunctionalityworks.Therearehoweverseveraldatasetsfreelyavailableonthewebthatarealotbigger.Wecollectedsomeofthemwithimportscriptssoyoumayplayaroundwiththem.AnotherhugegraphisthePokecsocialnetworkfromSlovakiathatweusedforperformancetestingonseveraldatabases;Youwillfindimportingscriptsetc.inthisblogpost.
Graphs
123
GraphsThischapterdescribesthegeneral-graphmodule.Itallowsyoutodefineagraphthatisspreadacrossseveraledgeanddocumentcollections.Thisallowsyoutostructureyourmodelsinlinewithyourdomainandgroupthemlogicallyincollectionsgivingyouthepowertoquerytheminthesamegraphqueries.Thereisnoneedtoincludethereferencedcollectionswithinthequery,thismodulewillhandleitforyou.
ThreeStepstocreateagraph
Createagraph
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>vargraph=graph_module._create("myGraph");
arangosh>graph;
{[Graph]
}
Addsomevertexcollections
arangosh>graph._addVertexCollection("shop");
arangosh>graph._addVertexCollection("customer");
arangosh>graph._addVertexCollection("pet");
arangosh>graph;
showexecutionresultsDefinerelationsontheGraph
arangosh>varrel=graph_module._relation("isCustomer",["shop"],["customer"]);
arangosh>graph._extendEdgeDefinitions(rel);
arangosh>graph;
showexecutionresults
GeneralGraphs
124
GraphManagementThischapterdescribesthejavascriptinterfaceforcreatingandmodifyingnamedgraphs.Inordertocreateanonemptygraphthefunctionalitytocreateedgedefinitionshastobeintroducedfirst:
EdgeDefinitions
Anedgedefinitionisalwaysadirectedrelationofagraph.Eachgraphcanhavearbitrarymanyrelationsdefinedwithintheedgedefinitionsarray.
Initializethelist
Createalistofedgedefinitionstoconstructagraph.
graph_module._edgeDefinitions(relation1,relation2,...,relationN)
Thelistofedgedefinitionsofagraphcanbemanagedbythegraphmoduleitself.Thisfunctionistheentrypointforthemanagementandwillreturnthecorrectlist.
Parameters
relationX(optional)Anobjectrepresentingadefinitionofonerelationinthegraph
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>directed_relation=graph_module._relation("lives_in","user","city");
arangosh>undirected_relation=graph_module._relation("knows","user","user");
arangosh>edgedefinitions=graph_module._edgeDefinitions(directed_relation,
undirected_relation);
showexecutionresults
Extendthelist
Extendthelistofedgedefinitionstoconstructagraph.
graph_module._extendEdgeDefinitions(edgeDefinitions,relation1,relation2,...,relationN)
Inordertoaddmoreedgedefinitionstothegraphbeforecreatingthisfunctioncanbeusedtoaddmoredefinitionstotheinitiallist.
Parameters
edgeDefinitions(required)Alistofrelationdefinitionobjects.relationX(required)Anobjectrepresentingadefinitionofonerelationinthegraph
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>directed_relation=graph_module._relation("lives_in","user","city");
arangosh>undirected_relation=graph_module._relation("knows","user","user");
arangosh>edgedefinitions=graph_module._edgeDefinitions(directed_relation);
arangosh>edgedefinitions=graph_module._extendEdgeDefinitions(undirected_relation);
showexecutionresults
Relation
Defineadirectedrelation.
GraphManagement
125
graph_module._relation(relationName,fromVertexCollections,toVertexCollections)
TherelationNamedefinesthenameofthisrelationandreferencestotheunderlyingedgecollection.ThefromVertexCollectionsisanArrayofdocumentcollectionsholdingthestartvertices.ThetoVertexCollectionsisanArrayofdocumentcollectionsholdingthetargetvertices.RelationsareonlyallowedinthedirectionfromanycollectioninfromVertexCollectionstoanycollectionintoVertexCollections.
Parameters
relationName(required)Thenameoftheedgecollectionwheretheedgesshouldbestored.Willbecreatedifitdoesnotyetexist.fromVertexCollections(required)Oneoralistofcollectionnames.Sourceverticesfortheedgeshavetobestoredinthesecollections.Collectionswillbecreatediftheydonotexist.toVertexCollections(required)Oneoralistofcollectionnames.Targetverticesfortheedgeshavetobestoredinthesecollections.Collectionswillbecreatediftheydonotexist.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._relation("has_bought",["Customer","Company"],["Groceries",
"Electronics"]);
showexecutionresults
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._relation("has_bought","Customer","Product");
showexecutionresults
CreateagraphAfterhavingintroducededgedefinitionsagraphcanbecreated.
Createagraph
graph_module._create(graphName,edgeDefinitions,orphanCollections)
Thecreationofagraphrequiresthenameofthegraphandadefinitionofitsedges.
Foreverytypeofedgedefinitionaconveniencemethodexiststhatcanbeusedtocreateagraph.Optionallyalistofvertexcollectionscanbeadded,whicharenotusedinanyedgedefinition.Thesecollectionsarereferredtoasorphancollectionswithinthischapter.Allcollectionsusedwithinthecreationprocessarecreatediftheydonotexist.
Parameters
graphName(required)UniqueidentifierofthegraphedgeDefinitions(optional)ListofrelationdefinitionobjectsorphanCollections(optional)Listofadditionalvertexcollectionnames
Examples
Createanemptygraph,edgedefinitionscanbeaddedatruntime:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph=graph_module._create("myGraph");
{[Graph]
}
Createagraphusinganedgecollectionedgesandasinglevertexcollectionvertices
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>varedgeDefinitions=[{collection:"edges","from":["vertices"],"to":[
"vertices"]}];
GraphManagement
126
arangosh>graph=graph_module._create("myGraph",edgeDefinitions);
{[Graph]
"edges":[ArangoCollection18495,"edges"(typeedge,statusloaded)],
"vertices":[ArangoCollection18490,"vertices"(typedocument,statusloaded)]
}
Createagraphwithedgedefinitionsandorphancollections:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph=graph_module._create("myGraph",
........>[graph_module._relation("myRelation",["male","female"],["male","female"])],
["sessions"]);
showexecutionresults
CompleteExampletocreateagraph
ExampleCall:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>varedgeDefinitions=graph_module._edgeDefinitions();
arangosh>graph_module._extendEdgeDefinitions(edgeDefinitions,
graph_module._relation("friend_of","Customer","Customer"));
arangosh>graph_module._extendEdgeDefinitions(
........>edgeDefinitions,graph_module._relation(
........>"has_bought",["Customer","Company"],["Groceries","Electronics"]));
arangosh>graph_module._create("myStore",edgeDefinitions);
showexecutionresultsalternativecall:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>varedgeDefinitions=graph_module._edgeDefinitions(
........>graph_module._relation("friend_of",["Customer"],["Customer"]),
graph_module._relation(
........>"has_bought",["Customer","Company"],["Groceries","Electronics"]));
arangosh>graph_module._create("myStore",edgeDefinitions);
showexecutionresults
Listavailablegraphs
Listallgraphs.
graph_module._list()
Listsallgraphnamesstoredinthisdatabase.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._list();
[]
Loadagraph
GraphManagement
127
Getagraph
graph_module._graph(graphName)
Agraphcanberetrievedbyitsname.
Parameters
graphName(required)Uniqueidentifierofthegraph
Examples
Getagraph:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph=graph_module._graph("social");
showexecutionresults
Removeagraph
Removeagraph
graph_module._drop(graphName,dropCollections)
Agraphcanbedroppedbyitsname.Thiscandropallcollectionscontainedinthegraphaslongastheyarenotusedwithinothergraphs.Todropthecollectionsonlybelongingtothisgraph,theoptionalparameterdrop-collectionshastobesettotrue.
Parameters
graphName(required)UniqueidentifierofthegraphdropCollections(optional)Defineifcollectionsshouldbedropped(default:false)
Examples
Dropagraphandkeepcollections:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._drop("social");
true
arangosh>db._collection("female");
[ArangoCollection18617,"female"(typedocument,statusloaded)]
arangosh>db._collection("male");
[ArangoCollection18620,"male"(typedocument,statusloaded)]
arangosh>db._collection("relation");
[ArangoCollection18623,"relation"(typeedge,statusloaded)]
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._drop("social",true);
true
arangosh>db._collection("female");
null
arangosh>db._collection("male");
null
arangosh>db._collection("relation");
null
ModifyagraphdefinitionduringruntimeAfteryouhavecreatedangraphitsdefinitionisnotimmutable.Youcanstilladd,deleteormodifyedgedefinitionsandvertexcollections.
GraphManagement
128
Extendtheedgedefinitions
Addanotheredgedefinitiontothegraph
graph._extendEdgeDefinitions(edgeDefinition)
Extendstheedgedefinitionsofagraph.Ifanorphancollectionisusedinthisedgedefinition,itwillberemovedfromtheorphanage.Iftheedgecollectionoftheedgedefinitiontoaddisalreadyusedinthegraphorusedinadifferentgraphwithdifferentfromand/ortocollectionsanerroristhrown.
Parameters
edgeDefinition(required)Therelationdefinitiontoextendthegraph
Examples
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vared2=graph_module._relation("myEC2",["myVC1"],["myVC3"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1]);
arangosh>graph._extendEdgeDefinitions(ed2);
Modifyanedgedefinition
Modifyanrelationdefinition
graph_module._editEdgeDefinition(edgeDefinition)
Editsonerelationdefinitionofagraph.Theedgedefinitionusedasargumentwillreplacetheexistingedgedefinitionofthegraphwhichhasthesamecollection.VertexCollectionsofthereplacededgedefinitionthatarenotusedinthenewdefinitionwilltransformtoanorphan.Orphansthatareusedinthisnewedgedefinitionwillbedeletedfromthelistoforphans.Othergraphswiththesameedgedefinitionwillbemodified,too.
Parameters
edgeDefinition(required)Theedgedefinitiontoreplacetheexistingedgedefinitionwiththesameattributecollection.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>varoriginal=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>varmodified=graph_module._relation("myEC1",["myVC2"],["myVC3"]);
arangosh>vargraph=graph_module._create("myGraph",[original]);
arangosh>graph._editEdgeDefinitions(modified);
Deleteanedgedefinition
Deleteonerelationdefinition
graph_module._deleteEdgeDefinition(edgeCollectionName,dropCollection)
Deletesarelationdefinitiondefinedbytheedgecollectionofagraph.Ifthecollectionsdefinedintheedgedefinition(collection,from,to)arenotusedinanotheredgedefinitionofthegraph,theywillbemovedtotheorphanage.
Parameters
edgeCollectionName(required)Nameofedgecollectionintherelationdefinition.dropCollection(optional)Defineiftheedgecollectionshouldbedropped.Defaultfalse.
Examples
Removeanedgedefinitionbutkeeptheedgecollection:
arangosh>vargraph_module=require("@arangodb/general-graph")
GraphManagement
129
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vared2=graph_module._relation("myEC2",["myVC1"],["myVC3"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1,ed2]);
arangosh>graph._deleteEdgeDefinition("myEC1");
arangosh>db._collection("myEC1");
[ArangoCollection25924,"myEC1"(typeedge,statusloaded)]
Removeanedgedefinitionanddroptheedgecollection:
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vared2=graph_module._relation("myEC2",["myVC1"],["myVC3"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1,ed2]);
arangosh>graph._deleteEdgeDefinition("myEC1",true);
arangosh>db._collection("myEC1");
null
ExtendvertexCollections
Eachgraphcanhaveanarbitraryamountofvertexcollections,whicharenotpartofanyedgedefinitionofthegraph.Thesecollectionsarecalledorphancollections.Ifthegraphisextendedwithanedgedefinitionusingoneoftheorphans,itwillberemovedfromthesetoforphancollectionautomatically.
Addavertexcollection
Addavertexcollectiontothegraph
graph._addVertexCollection(vertexCollectionName,createCollection)
Addsavertexcollectiontothesetoforphancollectionsofthegraph.Ifthecollectiondoesnotexist,itwillbecreated.Ifitisalreadyusedbyanyedgedefinitionofthegraph,anerrorwillbethrown.
Parameters
vertexCollectionName(required)Nameofvertexcollection.createCollection(optional)Iftruethecollectionwillbecreatedifitdoesnotexist.Default:true.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1]);
arangosh>graph._addVertexCollection("myVC3",true);
Gettheorphanedcollections
Getallorphancollections
graph._orphanCollections()
Returnsallvertexcollectionsofthegraphthatarenotusedinanyedgedefinition.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1]);
arangosh>graph._addVertexCollection("myVC3",true);
arangosh>graph._orphanCollections();
GraphManagement
130
[
"myVC3"
]
Removeavertexcollection
Removeavertexcollectionfromthegraph
graph._removeVertexCollection(vertexCollectionName,dropCollection)
Removesavertexcollectionfromthegraph.Onlycollectionsnotusedinanyrelationdefinitioncanberemoved.Optionallythecollectioncanbedeleted,ifitisnotusedinanyothergraph.
Parameters
vertexCollectionName(required)Nameofvertexcollection.dropCollection(optional)Iftruethecollectionwillbedroppedifitisnotusedinanyothergraph.Default:false.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1]);
arangosh>graph._addVertexCollection("myVC3",true);
arangosh>graph._addVertexCollection("myVC4",true);
arangosh>graph._orphanCollections();
arangosh>graph._removeVertexCollection("myVC3");
arangosh>graph._orphanCollections();
showexecutionresults
ManiuplatingVertices
Saveavertex
CreateanewvertexinvertexCollectionName
graph.vertexCollectionName.save(data)
Parameters
data(required)JSONdataofvertex.
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.male.save({name:"Floyd",_key:"floyd"});
showexecutionresults
Replaceavertex
ReplacesthedataofavertexincollectionvertexCollectionName
graph.vertexCollectionName.replace(vertexId,data,options)
Parameters
vertexId(required)_idattributeofthevertexdata(required)JSONdataofvertex.
GraphManagement
131
options(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.male.save({neym:"Jon",_key:"john"});
arangosh>graph.male.replace("male/john",{name:"John"});
showexecutionresults
Updateavertex
UpdatesthedataofavertexincollectionvertexCollectionName
graph.vertexCollectionName.update(vertexId,data,options)
Parameters
vertexId(required)_idattributeofthevertexdata(required)JSONdataofvertex.options(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.female.save({name:"Lynda",_key:"linda"});
arangosh>graph.female.update("female/linda",{name:"Linda",_key:"linda"});
showexecutionresults
Removeavertex
RemovesavertexincollectionvertexCollectionName
graph.vertexCollectionName.remove(vertexId,options)
Additionallyremovesallingoingandoutgoingedgesofthevertexrecursively(seeedgeremove).
Parameters
vertexId(required)_idattributeofthevertexoptions(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.male.save({name:"Kermit",_key:"kermit"});
arangosh>db._exists("male/kermit")
arangosh>graph.male.remove("male/kermit")
arangosh>db._exists("male/kermit")
showexecutionresults
ManipulatingEdges
Saveanewedge
GraphManagement
132
CreatesanedgefromvertexfromtovertextoincollectionedgeCollectionName
graph.edgeCollectionName.save(from,to,data,options)
Parameters
from(required)_idattributeofthesourcevertexto(required)_idattributeofthetargetvertexdata(required)JSONdataoftheedgeoptions(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save("male/bob","female/alice",{type:"married",_key:
"bobAndAlice"});
showexecutionresultsIfthecollectionsoffromandtoarenotdefinedinanedgedefinitionofthegraph,theedgewillnotbestored.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save(
........>"relation/aliceAndBob",
........>"female/alice",
........>{type:"married",_key:"bobAndAlice"});
[ArangoError1906:invalidedgebetweenrelation/aliceAndBobandfemale/alice.Doesn't
conformtoanyedgedefinition]
Replaceanedge
ReplacesthedataofanedgeincollectionedgeCollectionName.Notethat_fromand_toaremandatory.
graph.edgeCollectionName.replace(edgeId,data,options)
Parameters
edgeId(required)_idattributeoftheedgedata(required)JSONdataoftheedgeoptions(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save("female/alice","female/diana",{typo:"nose",_key:
"aliceAndDiana"});
arangosh>graph.relation.replace("relation/aliceAndDiana",{type:"knows",_from:
"female/alice",_to:"female/diana"});
showexecutionresults
Updateanedge
UpdatesthedataofanedgeincollectionedgeCollectionName
graph.edgeCollectionName.update(edgeId,data,options)
Parameters
GraphManagement
133
edgeId(required)_idattributeoftheedgedata(required)JSONdataoftheedgeoptions(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save("female/alice","female/diana",{type:"knows",_key:
"aliceAndDiana"});
arangosh>graph.relation.update("relation/aliceAndDiana",{type:"quarreled",_key:
"aliceAndDiana"});
showexecutionresults
Removeanedge
RemovesanedgeincollectionedgeCollectionName
graph.edgeCollectionName.remove(edgeId,options)
Ifthisedgeisusedasavertexbyanotheredge,theotheredgewillberemoved(recursively).
Parameters
edgeId(required)_idattributeoftheedgeoptions(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save("female/alice","female/diana",{_key:"aliceAndDiana"});
arangosh>db._exists("relation/aliceAndDiana")
arangosh>graph.relation.remove("relation/aliceAndDiana")
arangosh>db._exists("relation/aliceAndDiana")
showexecutionresults
Connectedges
Getallconnectingedgesbetween2groupsofverticesdefinedbytheexamples
graph._getConnectingEdges(vertexExample,vertexExample2,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.
Parameters
vertexExample1(optional)SeeDefinitionofexamplesvertexExample2(optional)SeeDefinitionofexamplesoptions(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
edgeExamples:Filtertheedges,seeDefinitionofexamplesedgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.vertex1CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertexsteps.vertex2CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertexsteps.
Examples
Arouteplannerexample,allconnectingedgesbetweencapitals.
GraphManagement
134
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._getConnectingEdges({isCapital:true},{isCapital:true});
[]
GraphManagement
135
GraphFunctionsThischapterdescribesvariousfunctionsonagraph.Alotoftheseacceptavertex(oredge)exampleasparameterasdefinedinthenextsection.
ExampleswillexplaintheAPIonthethecitygraph:
DefinitionofexamplesFormanyofthefollowingfunctionsexamplescanbepassedinasaparameter.Examplesareusedtofiltertheresultsetforobjectsthatmatchtheconditions.Theseexamplescanhavethefollowingvalues:
null,thereisnomatchingexecutedallfoundresultsarevalid.Astring,onlyresultsarereturned,which_idequalthevalueofthestringAnexampleobject,definingasetofattributes.Onlyresultshavingtheseattributesarematched.Alistcontainingexampleobjectsand/orstrings.Allresultsmatchingatleastoneoftheelementsinthelistarereturned.Getverticesfromedges.
Getvertexfromofanedge
Getthesourcevertexofanedge
graph._fromVertex(edgeId)
Returnsthevertexdefinedwiththeattribute_fromoftheedgewithedgeIdasits_id.
Parameters
edgeId(required)_idattributeoftheedge
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
GraphFunctions
136
arangosh>vargraph=examples.loadGraph("social");
arangosh>varany=require("@arangodb").db.relation.any();
arangosh>graph._fromVertex("relation/"+any._key);
showexecutionresults
Getvertextoofanedge
Getthetargetvertexofanedge
graph._toVertex(edgeId)
Returnsthevertexdefinedwiththeattribute_tooftheedgewithedgeIdasits_id.
Parameters
edgeId(required)_idattributeoftheedge
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>varany=require("@arangodb").db.relation.any();
arangosh>graph._toVertex("relation/"+any._key);
showexecutionresults
_neighbors
Getallneighborsoftheverticesdefinedbytheexample
graph._neighbors(vertexExample,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.ThecomplexityofthismethodisO(n*m^x)withnbeingtheverticesdefinedbytheparametervertexExamplex,mtheaverageamountofneighborsandxthemaximaldepths.HencethedefaultcallwouldhaveacomplexityofO(n*m);
Parameters
vertexExample(optional)SeeDefinitionofexamplesoptions(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).edgeExamples:Filtertheedges,seeDefinitionofexamplesneighborExamples:Filtertheneighborvertices,seeDefinitionofexamplesedgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.vertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertexsteps.minDepth:Definestheminimalnumberofintermediatestepstoneighbors(defaultis1).maxDepth:Definesthemaximalnumberofintermediatestepstoneighbors(defaultis1).
Examples
Arouteplannerexample,allneighborsofcapitals.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._neighbors({isCapital:true});
showexecutionresultsArouteplannerexample,alloutboundneighborsofHamburg.
GraphFunctions
137
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._neighbors('germanCity/Hamburg',{direction:'outbound',maxDepth:2});
showexecutionresults
_commonNeighbors
Getallcommonneighborsoftheverticesdefinedbytheexamples.
graph._commonNeighbors(vertex1Example,vertex2Examples,optionsVertex1,optionsVertex2)
Thisfunctionreturnstheintersectionofgraph_module._neighbors(vertex1Example,optionsVertex1)andgraph_module._neighbors(vertex2Example,optionsVertex2).Forparameterdocumentationsee_neighbors.
ThecomplexityofthismethodisO(n*m^x)withnbeingthemaximalamountofverticesdefinedbytheparametersvertexExamples,mtheaverageamountofneighborsandxthemaximaldepths.HencethedefaultcallwouldhaveacomplexityofO(n*m);
Examples
Arouteplannerexample,allcommonneighborsofcapitals.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._commonNeighbors({isCapital:true},{isCapital:true});
showexecutionresultsArouteplannerexample,allcommonoutboundneighborsofHamburgwithanyotherlocationwhichhaveamaximaldepthof2:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._commonNeighbors(
........>'germanCity/Hamburg',
........>{},
........>{direction:'outbound',maxDepth:2},
........>{direction:'outbound',maxDepth:2});
showexecutionresults
_countCommonNeighborsGettheamountofcommonneighborsoftheverticesdefinedbytheexamples.
graph._countCommonNeighbors(vertex1Example,vertex2Examples,optionsVertex1,optionsVertex2)
Similarto_commonNeighborsbutreturnscountinsteadoftheelements.
Examples
Arouteplannerexample,allcommonneighborsofcapitals.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>varexample={isCapital:true};
arangosh>varoptions={includeData:true};
arangosh>graph._countCommonNeighbors(example,example,options,options);
showexecutionresults
GraphFunctions
138
Arouteplannerexample,allcommonoutboundneighborsofHamburgwithanyotherlocationwhichhaveamaximaldepthof2:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>varoptions={direction:'outbound',maxDepth:2,includeData:true};
arangosh>graph._countCommonNeighbors('germanCity/Hamburg',{},options,options);
showexecutionresults
_commonPropertiesGettheverticesofthegraphthatsharecommonproperties.
graph._commonProperties(vertex1Example,vertex2Examples,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertex1Exampleandvertex2Example.
ThecomplexityofthismethodisO(n)withnbeingthemaximalamountofverticesdefinedbytheparametersvertexExamples.
Parameters
vertex1Examples(optional)Filterthesetofsourcevertices,seeDefinitionofexamples
vertex2Examples(optional)Filterthesetofverticescomparedto,seeDefinitionofexamples
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:vertex1CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbesearchedforsourcevertices.vertex2CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbesearchedforcomparevertices.ignoreProperties:Oneoralistofattributenamesofadocumentthatshouldbeignored.
Examples
Arouteplannerexample,alllocationswiththesameproperties:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._commonProperties({},{});
showexecutionresultsArouteplannerexample,allcitieswhichsharesamepropertiesexceptforpopulation.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._commonProperties({},{},{ignoreProperties:'population'});
showexecutionresults
_countCommonPropertiesGettheamountofverticesofthegraphthatsharecommonproperties.
graph._countCommonProperties(vertex1Example,vertex2Examples,options)
Similarto_commonPropertiesbutreturnscountinsteadoftheobjects.
Examples
Arouteplannerexample,alllocationswiththesameproperties:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
GraphFunctions
139
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._countCommonProperties({},{});
showexecutionresultsArouteplannerexample,allGermancitieswhichsharesamepropertiesexceptforpopulation.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._countCommonProperties({},{},{vertex1CollectionRestriction:
'germanCity',
........>vertex2CollectionRestriction:'germanCity',ignoreProperties:'population'});
showexecutionresults
_paths
The_pathsfunctionreturnsallpathsofagraph.
graph._paths(options)
Thisfunctiondeterminesallavailablepathsinagraph.
ThecomplexityofthismethodisO(n*n*m)withnbeingtheamountofverticesinthegraphandmtheaverageamountofconnectededges;
Parameters
options(optional)Anobjectcontainingoptions,seebelow:direction:Thedirectionoftheedges.Possiblevaluesareany,inboundandoutbound(default).followCycles(optional):Ifsettotruethequeryfollowscyclesinthegraph,defaultisfalse.minLength(optional):Definestheminimallengthapathmusthavetobereturned(defaultis0).maxLength(optional):Definesthemaximallengthapathmusthavetobereturned(defaultis10).
Examples
Returnallpathsofthegraph"social":
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("social");
arangosh>g._paths();
showexecutionresultsReturnallinboundpathsofthegraph"social"withamaximallengthof1andaminimallengthof2:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("social");
arangosh>g._paths({direction:'inbound',minLength:1,maxLength:2});
showexecutionresults
_shortestPath
The_shortestPathfunctionreturnsallshortestpathsofagraph.
graph._shortestPath(startVertexExample,endVertexExample,options)
GraphFunctions
140
Thisfunctiondeterminesallshortestpathsinagraph.Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforstartandendvertex.Thelengthofapathisbydefaulttheamountofedgesfromonestartvertextoanendvertex.Theoptionweightallowstheusertodefineanedgeattributerepresentingthelength.
Parameters
startVertexExample(optional)AnexampleforthedesiredstartVertices(seeDefinitionofexamples).endVertexExample(optional)AnexampleforthedesiredendVertices(seeDefinitionofexamples).options(optional)Anobjectcontainingoptions,seebelow:
direction:Thedirectionoftheedgesasastring.Possiblevaluesareoutbound,inboundandany(default).edgeCollectionRestriction:Oneormultipleedgecollectionnames.Onlyedgesfromthesecollectionswillbeconsideredforthepath.startVertexCollectionRestriction:Oneormultiplevertexcollectionnames.Onlyverticesfromthesecollectionswillbeconsideredasstartvertexofapath.endVertexCollectionRestriction:Oneormultiplevertexcollectionnames.Onlyverticesfromthesecollectionswillbeconsideredasendvertexofapath.weight:Thenameoftheattributeoftheedgescontainingthelengthasastring.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedaslength.IfnodefaultissuppliedthedefaultwouldbepositiveInfinitysothepathcouldnotbecalculated.
Examples
Arouteplannerexample,shortestpathfromallgermantoallfrenchcities:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>g._shortestPath({},{},{weight:'distance',endVertexCollectionRestriction:
'frenchCity',
........>startVertexCollectionRestriction:'germanCity'});
showexecutionresultsArouteplannerexample,shortestpathfromHamburgandColognetoLyon:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>g._shortestPath([{_id:'germanCity/Cologne'},{_id:'germanCity/Munich'}],
'frenchCity/Lyon',
........>{weight:'distance'});
showexecutionresults
_distanceTo
The_distanceTofunctionreturnsallpathsandtheredistancewithinagraph.
graph._distanceTo(startVertexExample,endVertexExample,options)
Thisfunctionisawrapperofgraph._shortestPath.Itdoesnotreturntheactualpathbutonlythedistancebetweentwovertices.
Examples
Arouteplannerexample,shortestdistancefromallgermantoallfrenchcities:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>g._distanceTo({},{},{weight:'distance',endVertexCollectionRestriction:
'frenchCity',
........>startVertexCollectionRestriction:'germanCity'});
GraphFunctions
141
showexecutionresultsArouteplannerexample,shortestdistancefromHamburgandColognetoLyon:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>g._distanceTo([{_id:'germanCity/Cologne'},{_id:'germanCity/Munich'}],
'frenchCity/Lyon',
........>{weight:'distance'});
showexecutionresults
_absoluteEccentricityGettheeccentricityoftheverticesdefinedbytheexamples.
graph._absoluteEccentricity(vertexExample,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.
Parameters
vertexExample(optional)Filterthevertices,seeDefinitionofexamplesoptions(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).edgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.startVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredforsourcevertices.endVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredfortargetvertices.weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheeccentricitycannotbecalculated.
Examples
Arouteplannerexample,theabsoluteeccentricityofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteEccentricity({});
showexecutionresultsArouteplannerexample,theabsoluteeccentricityofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteEccentricity({},{weight:'distance'});
showexecutionresultsArouteplannerexample,theabsoluteeccentricityofallcitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteEccentricity({},{startVertexCollectionRestriction:
'germanCity',
........>direction:'outbound',weight:'distance'});
showexecutionresults
GraphFunctions
142
_eccentricity
Getthenormalizedeccentricityoftheverticesdefinedbytheexamples.
graph._eccentricity(vertexExample,options)
Similarto_absoluteEccentricitybutreturnsanormalizedresult.
Examples
Arouteplannerexample,theeccentricityofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._eccentricity();
showexecutionresultsArouteplannerexample,theweightedeccentricity.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._eccentricity({weight:'distance'});
showexecutionresults
_absoluteCloseness
Gettheclosenessoftheverticesdefinedbytheexamples.
graph._absoluteCloseness(vertexExample,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.
Parameters
vertexExample(optional)Filterthevertices,seeDefinitionofexamplesoptions(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).edgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.startVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredforsourcevertices.endVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredfortargetvertices.weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheclosenesscannotbecalculated.
Examples
Arouteplannerexample,theabsoluteclosenessofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteCloseness({});
showexecutionresultsArouteplannerexample,theabsoluteclosenessofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
GraphFunctions
143
arangosh>graph._absoluteCloseness({},{weight:'distance'});
showexecutionresultsArouteplannerexample,theabsoluteclosenessofallGermanCitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteCloseness({},{startVertexCollectionRestriction:'germanCity',
........>direction:'outbound',weight:'distance'});
showexecutionresults
_closeness
Getthenormalizedclosenessofgraphsvertices.
graph._closeness(options)
Similarto_absoluteClosenessbutreturnsanormalizedvalue.
Examples
Arouteplannerexample,thenormalizedclosenessofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._closeness();
showexecutionresultsArouteplannerexample,theclosenessofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._closeness({weight:'distance'});
showexecutionresultsArouteplannerexample,theclosenessofallcitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._closeness({direction:'outbound',weight:'distance'});
showexecutionresults
_absoluteBetweennessGetthebetweennessofallverticesinthegraph.
graph._absoluteBetweenness(vertexExample,options)
Parameters
vertexExample(optional)Filterthevertices,seeDefinitionofexamplesoptions(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthis
GraphFunctions
144
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencethebetweenesscannotbecalculated.
Examples
Arouteplannerexample,theabsolutebetweennessofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteBetweenness({});
showexecutionresultsArouteplannerexample,theabsolutebetweennessofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteBetweenness({weight:'distance'});
{
}
Arouteplannerexample,theabsolutebetweennessofallcitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteBetweenness({direction:'outbound',weight:'distance'});
{
}
_betweenness
Getthenormalizedbetweennessofgraphsvertices.
graph_module._betweenness(options)
Similarto_absoluteBetweenessbutreturnsnormalizedvalues.
Examples
Arouteplannerexample,thebetweennessofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._betweenness();
showexecutionresultsArouteplannerexample,thebetweennessofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._betweenness({weight:'distance'});
showexecutionresultsArouteplannerexample,thebetweennessofallcitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._betweenness({direction:'outbound',weight:'distance'});
GraphFunctions
145
showexecutionresults
_radiusGettheradiusofagraph.
`
Parameters
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheradiuscannotbecalculated.
Examples
Arouteplannerexample,theradiusofthegraph.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._radius();
1
Arouteplannerexample,theradiusofthegraph.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._radius({weight:'distance'});
1
Arouteplannerexample,theradiusofthegraphregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._radius({direction:'outbound',weight:'distance'});
1
_diameter
Getthediameterofagraph.
graph._diameter(graphName,options)
Parameters
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).weight:Thenameoftheattributeoftheedgescontainingtheweight.defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthisdefaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheradiuscannotbecalculated.
Examples
GraphFunctions
146
Arouteplannerexample,thediameterofthegraph.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._diameter();
1
Arouteplannerexample,thediameterofthegraph.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._diameter({weight:'distance'});
1
Arouteplannerexample,thediameterofthegraphregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._diameter({direction:'outbound',weight:'distance'});
1
GraphFunctions
147
SmartGraphsThisfeatureisonlyavailableintheEnterpriseEdition.
Thischapterdescribesthesmart-graphmodule.Itenablesyoutomanagegraphsatscale,itwillgiveavastperformancebenefitforallgraphsshardedinanArangoDBCluster.Onasingleserverthisfeatureispointless,henceitisonlyavailableinaclustermode.IntermsofqueryingthereisnodifferencebetweensmartandGeneralGraphs.Theformerareatransparentreplacementforthelatter.SoforqueryingthegraphpleaserefertoAQLGraphOperationsandGraphFunctionssections.TheoptimizeriscleverenoughtoidentifyifweareonaSmartGraphornot.
Thedifferenceisonlyinthemanagementsection:creatingandmodifyingtheunderlyingcollectionsofthegraph.ForadetailedAPIreferencepleaserefertoSmartGraphManagement.
Whatmakesagraphsmart?
Mostgraphshaveonefeaturethatdividestheentiregraphintoseveralsmallersubgraphs.Thesesubgraphshavealargeamountofedgesthatonlyconnectverticesinthesamesubgraphandonlyhavefewedgesconnectingverticesfromothersubgraphs.Examplesforthesegraphsare:
SocialNetworks
Typicallythefeaturehereistheregion/countryuserslivein.Everyusertypicalliyhasmorecontactsinthesameregion/countrythenshehasinotherregions/countries
TransportSystems
Forthosealsothefeatureistheregion/country.Youhavemanylocaltransportionbutonlyfewaccrosscountries.
E-Commerce
Inthiscaseprobablythecategoryofproductsisagoodfeature.Oftenproductsofthesamecategoryareboughttogether.
Ifthisfeatureisknown,SmartGraphscanmakeuseifit.WhencreatingaSmartGraphyouhavetodefineasmartAttribute,whichisthenameofanattributestoredineveryvertex.Thegraphwillthanbeautomaticallyshardedinsuchawaythatallverticeswiththesamevaluearestoredonthesamephysicalmachine,alledgesconnectingverticeswithidenticalsmartAttributevaluesarestoredonthismachineaswell.Duringquerytimethequeryoptimizerandthequeryexecutorbothknowforeverydocumentexactlywhereitisstoredandcantherebyminimizenetworkoverhead.Everythingthatcanbecomputedlocallywillbecomputedlocally.
BenefitsofSmartGraphs
Becauseoftheabovedescribedguaranteedsharding,theperformanceofqueriesthatonlycoveronesubgraphhaveaperformancealmostequaltoanonlylocalcomputation.Queriesthatcovermorethanonesubgraphrequiresomenetworkoverhead.Themoresubgraphsaretouchedthemorenetworkcostwillapply.HowevertheoverallperformanceisneverworsethanthesamequeryonaGeneralGraph.
Gettingstarted
FirstofallSmartGraphscannotuseexistingcollections,whenswitchingtoSmartGraphfromanexistingdatasetyouhavetoimportthedataintoafreshSmartGraph.Thisswitchcanbeeasilyachievedwitharangodumpandarangorestore.TheonlythingyouhavetochangeinthispipelineisthatyoucreatethenewcollectionswiththeSmartGraphbeforestartingarangorestore.
Createagraph
IncomparisontoGeneralGraphwehavetoaddmoreoptionswhencreatingthegraph.ThetwooptionssmartGraphAttributeandnumberOfShardsarerequiredandcannotbemodifedlater.
arangosh>vargraph_module=require("@arangodb/smart-graph");
arangosh>vargraph=graph_module._create("myGraph",[],[],{smartGraphAttribute:"region",numberOfShards:9});
arangosh>graph;
[SmartGraphmyGraphEdgeDefinitions:[]VertexCollections:[]]
Addsomevertexcollections
SmartGraphs
148
ThisisagainidenticaltoGeneralGraph.Themodulewillsetupcorrectshardingforallthesecollections.Note:Thecollectionshavetobenew.
arangosh>graph._addVertexCollection("shop");
arangosh>graph._addVertexCollection("customer");
arangosh>graph._addVertexCollection("pet");
arangosh>graph;
[SmartGraphmyGraphEdgeDefinitions:[]VertexCollections:["shop","customer","pet"]]
DefinerelationsontheGraph
arangosh>varrel=graph_module._relation("isCustomer",["shop"],["customer"]);
arangosh>graph._extendEdgeDefinitions(rel);
arangosh>graph;
[SmartGraphmyGraphEdgeDefinitions:["isCustomer:[shop]->[customer]"]VertexCollections:["pet"]]
SmartGraphs
149
SmartGraphManagementThischapterdescribestheJavaScriptinterfaceforcreatingandmodifyingSmartGraphs.AtfirstyouhavetonotethateverySmartGraphisaspecializedversionofaGeneralGraph,whichmeansalloftheGeneralGraphfunctionalityisavailableonaSmartGraphaswell.Themajordifferenceofbothmodulesishandlingoftheunderlyingcollections,theGeneralGraphdoesnotenforceormaintainanyshardingofthecollectionsandcanthereforcombinearbitrarysetsofexistingcollections.SmartGraphsenforceandrelyonaspecialshardingoftheunderlyingcollectionsandhencecanonlyworkwithcollectionsthatarecreatedthroughtheSmartGraphitself.ThisalsomeansthatSmartGraphscannotbeoverlapping,acollectioncaneitherbeshardedforoneSmartGraphorfortheother.IfyouneedtomakesurethatallqueriescanbeexecutedwithSmartGraphperformance,justcreateonelargeSmartGraphcoveringeverythingandqueryitstatingthesubsetofedgecollectionsexplicitly.TogenerallyunderstandtheconceptofthismodulepleasereadthechapteraboutGeneralGraphManagementfirst.Inthefollowingwewillonlydescribetheoverloadedfunctionality.Everythingelseworksidenticalinbothmodules.
Createagraph
AlsoSmartGraphsrequireedgerelationstobecreated,theformatoftherelationsisidentical.TheonlydifferenceisthatallcollectionsusedwithintherelationstocreateanewSmartGraphcannotexistyet.TheyhavetobecreatedbytheGraphinordertoenforcethecorrectsharding.
Createagraph
graph_module._create(graphName,edgeDefinitions,orphanCollections,smartOptions)
ThecreationofagraphrequiresthenameandsomeSmartGraphoptions.DuetotheAPIedgeDefinitionsandorphanCollectionshavetobegiven,butbothcanbeemptyarraysandcanbecreatedlater.TheedgeDefinitionscanbecreatedusingtheconveniencemethod_relationknownfromthegeneral-graphmodule,whichisalsoavailablehere.orphanCollectionsagainisjustalistofadditionalvertexcollectionswhicharenotyetconnectedviaedgesbutshouldfollowthesameshardingtobeconnectedlateron.Allcollectionsusedwithinthecreationprocessarenewlycreated.Theprocesswillfailifoneofthemalreadyexists.Allnewlycreatedcollectionswillimmediatelybedroppedagaininthefailedcase.
Parameters
graphName(required)UniqueidentifierofthegraphedgeDefinitions(required)Listofrelationdefinitionobjects,maybeemptyorphanCollections(required)Listofadditionalvertexcollectionnames,maybeemptysmartOptions(required)AJSONobjecthavingthefollowingkeys:
numberOfShards(required)Thenumberofshardsthatwillbecreatedforeachcollection.Tomaintainthecorrectshardingallcollectionsneedanidenticalnumberofshards.Thiscannotbemodifiedaftercreationofthegraph.smartGraphAttribute(required)Theattributethatwillbeusedforsharding.Allverticesarerequiredtohavethisattributesetandithastobeastring.Edgesderivetheattributefromtheirconnectedvertices.
Examples
Createanemptygraph,edgedefinitionscanbeaddedatruntime:
arangosh>vargraph_module=require("@arangodb/smart-graph");
arangosh>vargraph=graph_module._create("myGraph",[],[],{smartGraphAttribute:"region",numberOfShards:9});
[SmartGraphmyGraphEdgeDefinitions:[]VertexCollections:[]]
Createagraphusinganedgecollectionedgesandasinglevertexcollectionvertices
arangosh>vargraph_module=require("@arangodb/smart-graph");
arangosh>varedgeDefinitions=[graph_module._relation("edges","vertices","vertices")];
arangosh>vargraph=graph_module._create("myGraph",edgeDefinitions,[],{smartGraphAttribute:"region",numberOfShards:9});
[SmartGraphmyGraphEdgeDefinitions:["edges:[vertices]->[vertices]"]VertexCollections:[]]
Createagraphwithedgedefinitionsandorphancollections:
arangosh>vargraph_module=require("@arangodb/smart-graph");
arangosh>varedgeDefinitions=[graph_module._relation("myRelation",["male","female"],["male","female"])];
SmartGraphManagement
150
arangosh>vargraph=graph_module._create("myGraph",edgeDefinitions,["sessions"],{smartGraphAttribute:"region",numberOfSh
ards:9});
[GraphmyGraphEdgeDefinitions:[
"myRelation:[female,male]->[female,male]"
]VertexCollections:[
"sessions"
]]
ModifyagraphdefinitionduringruntimeAfteryouhavecreatedaSmartGraphitsdefinitionisalsonotimmutable.Youcanstilladdorremoverelations.ThisisagainidenticaltoGeneralGraphs.Howeverthereisoneimportantdifference:Youcanonlyaddcollectionsthateitherdonotexist,orthathavebeencreatedbythisgraphearlier.Thelatercanbeachievedifyouforexampleremoveanorphancollectionfromthisgraph,withoutdroppingthecollectionitself.Thanaftersometimeyoudecidetoadditagain,itcanbeused.Thisisbecausetheenforcedshardingisstillappliedtothisvertexcollection,henceitissuitabletobeaddedagain.
Removeavertexcollection
Removeavertexcollectionfromthegraph
graph._removeVertexCollection(vertexCollectionName,dropCollection)
InmostcasesthisfunctionworksidenticallytotheGeneralGraphone.Butthereisonespecialcase:Thefirstvertexcollectionaddedtothegraph(eitherorphanorwithinarelation)definestheshardingforallcollectionswithinthegraph.Thiscollectioncanneverberemovedfromthegraph.
Parameters
vertexCollectionName(required)Nameofvertexcollection.dropCollection(optional)Iftruethecollectionwillbedroppedifitisnotusedinanyothergraph.Default:false.
Examples
Thefollowingexampleshowsthatyoucannotdroptheinitialcollection.Youhavetodropthecompletegraph.Ifyoujustwanttogetridofthedatatruncateit.
arangosh>vargraph_module=require("@arangodb/smart-graph")
arangosh>varrelation=graph_module._relation("edges","vertices","vertices");
arangosh>vargraph=graph_module._create("myGraph",[relation],["other"],{smartGraphAttribute:"region",numberOfShards:
9});
arangosh>graph._orphanCollections();
[
"other"
]
arangosh>graph._deleteEdgeDefinition("edges");
arangosh>graph._orphanCollections();
[
"vertices",
"other"
]
arangosh>graph._removeVertexCollection("other");
arangosh>graph._orphanCollections();
[
"vertices"
]
arangosh>graph._removeVertexCollection("vertices");
ArangoError4002:cannotdropthissmartcollection
SmartGraphManagement
151
TraversalsArangoDBprovidesseveralwaystoquerygraphdata.Verysimpleoperationscanbecomposedwiththelow-leveledgemethodsedges,inEdges,andoutEdgesforedgecollections.Theseworkonnamedandanonymousgraphs.Formorecomplexoperations,ArangoDBprovidespredefinedtraversalobjects.
AlsoTraversalshavebeenaddedtoAQL.PleasereadthechapteraboutAQLtraversersalsbeforeyoucontinuereadinghere.MostofthetraversalcasesarecoveredbyAQLandwillbeexecutedinanoptimizedway.OnlyifthelogicforyouristoocomplextobedefinedusingAQLfiltersyoucanusethetraversalobjectdefinedherewhichgivesyoucompleteprogrammaticaccesstothedata.
Foranyofthefollowingexamples,we'llbeusingtheexamplecollectionsvande,populatedwithcontinents,countriesandcapitalsdatalistedbelow(seeExampleData).
StartingfromScratchArangoDBprovidestheedges,inEdges,andoutEdgesmethodsforedgecollections.Thesemethodscanbeusedtoquicklydetermineifavertexisconnectedtoothervertices,andwhich.ThisfunctionalitycanbeexploitedtowriteverysimplegraphqueriesinJavaScript.
Forexample,todeterminewhichedgesarelinkedtotheworldvertex,wecanuseinEdges:
db.e.inEdges('v/world').forEach(function(edge){
require("@arangodb").print(edge._from,"->",edge.type,"->",edge._to);
});
inEdgeswillgiveusallingoingedgesforthespecifiedvertexv/world.TheresultisaJavaScriptarraythatwecaniterateoverandprinttheresults:
v/continent-africa->is-in->v/world
v/continent-south-america->is-in->v/world
v/continent-asia->is-in->v/world
v/continent-australia->is-in->v/world
v/continent-europe->is-in->v/world
v/continent-north-america->is-in->v/world
Note:edges,inEdges,andoutEdgesreturnanarrayofedges.Ifwewanttoretrievethelinkedvertices,wecanuseeachedges'_fromand_toattributesasfollows:
db.e.inEdges('v/world').forEach(function(edge){
require("@arangodb").print(db._document(edge._from).name,"->",edge.type,"->",db._document(edge._to).name);
});
Weareusingthedocumentmethodfromthedbobjecttoretrievetheconnectedverticesnow.
Whilethismaybesufficientforone-levelgraphoperations,writingatraversalbyhandmaybecometoocomplexformulti-leveltraversals.
Traversals
152
GettingstartedTouseatraversalobject,wefirstneedtorequirethetraversalmodule:
vartraversal=require("@arangodb/graph/traversal");
varexamples=require("@arangodb/graph-examples/example-graph.js");
examples.loadGraph("worldCountry");
Wethenneedtosetupaconfigurationforthetraversalanddetermineatwhichvertextostartthetraversal:
varconfig={
datasource:traversal.generalGraphDatasourceFactory("worldCountry"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.inboundExpander,
maxDepth:1
};
varstartVertex=db._document("v/world");
Note:ThestartVertexneedstobeadocument,notonlyadocumentid.
Wecanthencreateatraverserandstartthetraversalbycallingitstraversemethod.Notethattraverseneedsaresultobject,whichitcanmodifyinplace:
varresult={
visited:{
vertices:[],
paths:[]
}
};
vartraverser=newtraversal.Traverser(config);
traverser.traverse(result,startVertex);
Finally,wecanprintthecontentsoftheresultsobject,limitedtothevisitedvertices.Wewillonlyprintthenameandtypeofeachvisitedvertexforbrevity:
require("@arangodb").print(result.visited.vertices.map(function(vertex){
returnvertex.name+"("+vertex.type+")";
}));
Thefullscript,whichincludesallstepscarriedoutsofaristhus:
vartraversal=require("@arangodb/graph/traversal");
varconfig={
datasource:traversal.generalGraphDatasourceFactory("worldCountry"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.inboundExpander,
maxDepth:1
};
varstartVertex=db._document("v/world");
varresult={
visited:{
vertices:[],
paths:[]
}
};
vartraverser=newtraversal.Traverser(config);
traverser.traverse(result,startVertex);
UsingTraversalObjects
153
require("@arangodb").print(result.visited.vertices.map(function(vertex){
returnvertex.name+"("+vertex.type+")";
}));
Theresultisanarrayofverticesthatwerevisitedduringthetraversal,startingatthestartvertex(i.e.v/worldinourexample):
[
"World(root)",
"Africa(continent)",
"Asia(continent)",
"Australia(continent)",
"Europe(continent)",
"NorthAmerica(continent)",
"SouthAmerica(continent)"
]
Note:Theresultislimitedtoverticesdirectlyconnectedtothestartvertex.WeachievedthisbysettingthemaxDepthattributeto1.Notsettingitwouldreturnthefullarrayofvertices.
TraversalDirection
Fortheexamplescontainedinthismanual,we'llbestartingthetraversalsatvertexv/world.Verticesinourgraphareconnectedlikethis:
v/world<-is-in<-continent(Africa)<-is-in<-country(Algeria)<-is-in<-capital(Algiers)
Togetanymeaningfulresults,wemusttraversethegraphininboundorder.Thismeans,we'llbefollowingallincomingedgesoftoavertex.Inthetraversalconfiguration,wehavespecifiedthisviatheexpanderattribute:
varconfig={
...
expander:traversal.inboundExpander
};
Forothergraphs,wemightwanttotraverseviatheoutgoingedges.Forthis,wecanusetheoutboundExpander.ThereisalsoananyExpander,whichwillfollowbothoutgoingandincomingedges.Thisshouldbeusedwithcareandthetraversalshouldalwaysbelimitedtoamaximumnumberofiterations(e.g.usingthemaxIterationsattribute)inordertoterminateatsomepoint.
Toinvokethedefaultoutboundexpanderforagraph,simplyusethepredefinedfunction:
varconfig={
...
expander:traversal.outboundExpander
};
Pleasenotetheoutboundexpanderwillnotproduceanyoutputfortheexamplesifwestillstartthetraversalatthev/worldvertex.
Still,wecanusetheoutboundexpanderifwestartsomewhereelseinthegraph,e.g.
vartraversal=require("@arangodb/graph/traversal");
varconfig={
datasource:traversal.generalGraphDatasourceFactory("world_graph"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.outboundExpander
};
varstartVertex=db._document("v/capital-algiers");
varresult={
visited:{
vertices:[],
paths:[]
}
};
UsingTraversalObjects
154
vartraverser=newtraversal.Traverser(config);
traverser.traverse(result,startVertex);
require("@arangodb").print(result.visited.vertices.map(function(vertex){
returnvertex.name+"("+vertex.type+")";
}));
Theresultis:
[
"Algiers(capital)",
"Algeria(country)",
"Africa(continent)",
"World(root)"
]
whichconfirmsthatnowwe'regoingoutbound.
TraversalStrategy
Depth-firsttraversals
Thevisitationorderofverticesisdeterminedbythestrategyandorderattributessetintheconfiguration.Wechosedepthfirstandpreorder,meaningthetraverserwillvisiteachvertexbeforehandlingconnectededges(pre-order),anddescendintoanyconnectededgesbeforeprocessingotherverticesonthesamelevel(depth-first).
Let'sremovethemaxDepthattributenow.We'llnowbegettingallvertices(directlyandindirectlyconnectedtothestartvertex):
varconfig={
datasource:traversal.generalGraphDatasourceFactory("world_graph"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.inboundExpander
};
varresult={
visited:{
vertices:[],
paths:[]
}
};
vartraverser=newtraversal.Traverser(config);
traverser.traverse(result,startVertex);
require("@arangodb").print(result.visited.vertices.map(function(vertex){
returnvertex.name+"("+vertex.type+")";
}));
Theresultwillbealongerarray,assembledindepth-first,pre-orderorder.Foreachcontinentfound,thetraverserwilldescendintolinkedcountries,andthenintothelinkedcapital:
[
"World(root)",
"Africa(continent)",
"Algeria(country)",
"Algiers(capital)",
"Angola(country)",
"Luanda(capital)",
"Botswana(country)",
"Gaborone(capital)",
"BurkinaFaso(country)",
"Ouagadougou(capital)",
...
]
UsingTraversalObjects
155
Let'sswitchtheorderattributefrompreordertopostorder.Thiswillmakethetraverservisitverticesafterallconnectedverticeswerevisited(i.e.mostdistantverticeswillbeemittedfirst):
[
"Algiers(capital)",
"Algeria(country)",
"Luanda(capital)",
"Angola(country)",
"Gaborone(capital)",
"Botswana(country)",
"Ouagadougou(capital)",
"BurkinaFaso(country)",
"Bujumbura(capital)",
"Burundi(country)",
"Yaounde(capital)",
"Cameroon(country)",
"N'Djamena(capital)",
"Chad(country)",
"Yamoussoukro(capital)",
"Coted'Ivoire(country)",
"Cairo(capital)",
"Egypt(country)",
"Asmara(capital)",
"Eritrea(country)",
"Africa(continent)",
...
]
Breadth-firsttraversals
Ifwegobacktopreorder,butchangethestrategytobreadth-firstandre-runthetraversal,we'llseethatthereturnorderchanges,anditemsonthesamelevelwillbereturnedadjacently:
[
"World(root)",
"Africa(continent)",
"Asia(continent)",
"Australia(continent)",
"Europe(continent)",
"NorthAmerica(continent)",
"SouthAmerica(continent)",
"BurkinaFaso(country)",
"Burundi(country)",
"Cameroon(country)",
"Chad(country)",
"Algeria(country)",
"Angola(country)",
...
]
Note:Theorderofitemsreturnedforthesamelevelisundefined.Thisisbecausethereisnonaturalorderofedgesforavertexwithmultipleconnectededges.Toexplicitlysettheorderforedgesonthesamelevel,youcanspecifyanedgecomparatorfunctionwiththesortattribute:
varconfig={
...
sort:function(l,r){returnl._key<r._key?1:-1;}
...
};
Theargumentslandrareedgedocuments.Thiswilltraverseedgesofthesamevertexinbackward_keyorder:
[
"World(root)",
"SouthAmerica(continent)",
"NorthAmerica(continent)",
"Europe(continent)",
"Australia(continent)",
"Asia(continent)",
UsingTraversalObjects
156
"Africa(continent)",
"Ecuador(country)",
"Colombia(country)",
"Chile(country)",
"Brazil(country)",
"Bolivia(country)",
"Argentina(country)",
...
]
Note:Thisattributeonlyworksfortheusualexpanderstraversal.inboundExpander,traversal.outboundExpander,traversal.anyExpanderandtheircorresponding"WithLabels"variants.Ifyouareusingcustomexpandersyouhavetoorganizethesortingwithinthespecifiedexpander.
WritingCustomVisitors
Sofarwehaveusedmuchofthetraverser'sdefaultfunctions.Thetraverserisveryconfigurableandmanyofthedefaultfunctionscanbeoverriddenwithcustomfunctionality.
Forexample,wehavebeenusingthedefaultvisitorfunction(whichisalwaysusediftheconfigurationdoesnotcontainthevisitorattribute).Thedefaultvisitorfunctioniscalledforeachvertexinatraversal,andwillpushitintotheresult.Thisisthereasonwhytheresultvariablelookeddifferentafterthetraversal,andneededtobeinitializedbeforethetraversalwasstarted.
Notethatthedefaultvisitor(namedtrackingVisitor)willaddeveryvisitedvertexintotheresult,includingthefullpathsfromthestartvertex.Thisisusefulforlearninganddebuggingpurposes,butshouldbeavoidedinproductionbecauseitmightproduce(andcopy)hugeamountsofdata.Instead,onlythosedatashouldbecopiedintotheresultthatareactuallynecessary.
Thetraversercomeswiththefollowingpredefinedvisitors:
trackingVisitor:thisisthedefaultvisitor.Itwillcopyalldataofeachvisitedvertexplusthefullpathinformationintotheresult.Thiscanbeslowiftheresultsetishugeorverticescontainalotofdata.countingVisitor:thisisaverylightweightvisitor:allitdoesisincreaseacounterintheresultforeachvertexvisited.Vertexdataandpathswillnotbecopiedintotheresult.doNothingVisitor:ifnoactionshallbecarriedoutwhenavertexisvisited,thisvisitorcanbeemployed.Itwillnotdoanythingandwillthusbefast.Itcanbeusedforperformancecomparisonswithothervisitors.
Wecanalsowriteourownvisitorfunctionifwewantto.Thegeneralfunctionsignatureforvisitorfunctionsisasfollows:
varconfig={
...
visitor:function(config,result,vertex,path,connected){...}
};
Note:theconnectedparametervaluewillonlybesetifthetraversalorderissettopreorder-expander.Otherwise,thisparameterwon'tbesetbythetraverser.
Visitorfunctionsarenotexpectedtoreturnanyvalues.Instead,theycanmodifytheresultvariable(e.g.bypushingthecurrentvertexintoit),ordoanythingelse.Forexample,wecancreateasimplevisitorfunctionthatonlyprintsinformationaboutthecurrentvertexaswetraverse:
varconfig={
datasource:traversal.generalGraphDatasourceFactory("world_graph"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.inboundExpander,
visitor:function(config,result,vertex,path){
require("@arangodb").print("visitingvertex",vertex.name);
}
};
vartraverser=newtraversal.Traverser(config);
traverser.traverse(undefined,startVertex);
Towriteavisitorthatincrementsacountereachtimeavertexisvisited,wecouldwritethefollowingcustomvisitor:
UsingTraversalObjects
157
config.visitor=function(config,result,vertex,path,connected){
if(!result){
result={};
}
if(!result.hasOwnProperty('count')){
result.count=0;
}
++result.count;
}
Notethatsuchvisitorisalreadypredefined(it'sthecountingVisitordescribedabove).Itcanbeusedasfollows:
config.visitor=traversal.countingVisitor;
Anotherexampleofavisitorisonethatcollectsthe_idvaluesofallverticesvisited:
config.visitor=function(config,result,vertex,path,connected){
if(!result){
result={};
}
if(!result.hasOwnProperty("visited")){
result.visited={vertices:[]};
}
result.visited.vertices.push(vertex._id);
}
Whenthetraversalorderissettopreorder-expander,thetraverserwillpassafifthparametervalueintothevisitorfunction.Thisparametercontainstheconnectededgesofthevisitedvertexasanarray.Thiscanbehandybecauseinthiscasethevisitorwillgetallinformationaboutthevertexandtheconnectededgestogether.
Forexample,thefollowingvisitorcanbeusedtoprintonlyleafnodes(thatdonothaveanyfurtherconnectededges):
config.visitor=function(config,result,vertex,path,connected){
if(connected&&connected.length===0){
require("@arangodb").print("foundaleaf-node:",vertex);
}
}
Notethatforthisvisitortowork,thetraversalorderattributeneedstobesettothevaluepreorder-expander.
FilteringVerticesandEdges
FilteringVertices
Sofarwehaveprintedorreturnedallverticesthatwerevisitedduringthetraversal.Thisisnotalwaysrequired.Iftheresultshallberestricttojustspecificvertices,wecanuseafilterfunctionforvertices.Itcanbedefinedbysettingthefilterattributeofatraversalconfiguration,e.g.:
varconfig={
filter:function(config,vertex,path){
if(vertex.type!=='capital'){
return'exclude';
}
}
}
Theabovefilterfunctionwillexcludeallverticesthatdonothaveatypevalueofcapital.Thefilterfunctionwillbecalledforeachvertexfoundduringthetraversal.Itwillreceivethetraversalconfiguration,thecurrentvertex,andthefullpathfromthetraversalstartvertextothecurrentvertex.Thepathconsistsofanarrayofedges,andanarrayofvertices.Wecouldalsofiltereverythingbutcapitalsby
UsingTraversalObjects
158
checkingthelengthofthepathfromthestartvertextothecurrentvertex.Capitalswillhaveadistanceof3fromthev/worldstartvertex(capital→is-in→country→is-in→continent→is-in→world):
varconfig={
...
filter:function(config,vertex,path){
if(path.edges.length<3){
return'exclude';
}
}
}
Note:Ifafilterfunctionreturnsnothing(orundefined),thecurrentvertexwillbeincluded,andallconnectededgeswillbefollowed.Ifafilterfunctionreturnsexcludethecurrentvertexwillbeexcludedfromtheresult,andallstillallconnectededgeswillbefollowed.Ifafilterfunctionreturnsprune,thecurrentvertexwillbeincluded,butnoconnectededgeswillbefollowed.
Forexample,thefollowingfilterfunctionwillnotdescendintoconnectededgesofcontinents,limitingthedepthofthetraversal.Still,continentverticeswillbeincludedintheresult:
varconfig={
...
filter:function(config,vertex,path){
if(vertex.type==='continent'){
return'prune';
}
}
}
Itisalsopossibletocombineexcludeandprunebyreturninganarraywithbothvalues:
return['exclude','prune'];
FilteringEdges
Itispossibletoexcludecertainedgesfromthetraversal.Tofilteronedges,afilterfunctioncanbedefinedviatheexpandFilterattribute.TheexpandFilterisafunctionwhichiscalledforeachedgeduringatraversal.
Itwillreceivethecurrentedge(edgevariable)andthevertexwhichtheedgeconnectsto(inthedirectionofthetraversal).Italsoreceivesthecurrentpathfromthestartvertexuptothecurrentvertex(excludingthecurrentedgeandthevertextheedgepointsto).
Ifthefunctionreturnstrue,theedgewillbefollowed.Ifthefunctionreturnsfalse,theedgewillnotbefollowed.Hereisaverysimplecustomedgefilterfunctionimplementation,whichsimplyincludesedgesifthe(edges)pathlengthislessthan1,andwillexcludeanyotheredges.Thiswilleffectivelyterminatethetraversalafterthefirstlevelofedges:
varconfig={
...
expandFilter:function(config,vertex,edge,path){
return(path.edges.length<1);
}
};
WritingCustomExpanders
Theedgesconnectedtoavertexaredeterminedbytheexpander.Sofarwehaveusedadefaultexpander(thedefaultinboundexpandertobeprecise).Thedefaultinboundexpandersimplyenumeratesallconnectedingoingedgesforavertex,basedontheedgecollectionspecifiedinthetraversalconfiguration.
Thereisalsoadefaultoutboundexpander,whichwillenumerateallconnectedoutgoingedges.Finally,thereisananyexpander,whichwillfollowbothingoingandoutgoingedges.
Ifconnectededgesmustbedeterminedinsomedifferentfashionforwhateverreason,acustomexpandercanbewrittenandregisteredbysettingtheexpanderattributeoftheconfiguration.Theexpanderfunctionsignatureisasfollows:
UsingTraversalObjects
159
varconfig={
...
expander:function(config,vertex,path){...}
}
Itistheexpander'sresponsibilitytoreturnalledgesandverticesdirectlyconnectedtothecurrentvertex(whichispassedviathevertexvariable).Thefullpathfromthestartvertexuptothecurrentvertexisalsosuppliedviathepathvariable.Anexpanderisexpectedtoreturnanarrayofobjects,whichneedtohaveanedgeandavertexattributeeach.
Note:Ifyouwanttorelyonaparticularorderinwhichtheedgesaretraversed,youhavetosorttheedgesreturnedbyyourexpanderwithinthecodeoftheexpander.Thefunctionstogetoutbound,inboundoranyedgesfromavertexdonotguaranteeanyparticularorder!
Acustomimplementationofaninboundexpandercouldlooklikethis(thisisanon-deterministicexpander,whichrandomlydecideswhetherornottoincludeconnectededges):
varconfig={
...
expander:function(config,vertex,path){
varconnected=[];
vardatasource=config.datasource;
datasource.getInEdges(vertex._id).forEach(function(edge){
if(Math.random()>=0.5){
connected.push({edge:edge,vertex:(edge._from)});
}
});
returnconnected;
}
};
Acustomexpandercanalsobeusedasanedgefilterbecauseithasfullcontroloverwhichedgeswillbereturned.
Followingaretwoexamplesofcustomexpandersthatpickedgesbasedonattributesoftheedgesandtheconnectedvertices.
Findingtheconnectededges/verticesbasedonanattributewhenintheconnectedvertices.Thegoalistofollowtheedgethatleadstothevertexwiththehighestvalueinthewhenattribute:
varconfig={
...
expander:function(config,vertex,path){
vardatasource=config.datasource;
//determinealloutgoingedges
varoutEdges=datasource.getOutEdges(vertex);
if(outEdges.length===0){
return[];
}
vardata=[];
outEdges.forEach(function(edge){
data.push({edge:edge,vertex:datasource.getInVertex(edge)});
});
//sortoutgoingverticesaccordingto"when"attributevalue
data.sort(function(l,r){
if(l.vertex.when===r.vertex.when){
return0;
}
return(l.vertex.when<r.vertex.when?1:-1);
});
//pickfirstvertexfound(withhighest"when"attributevalue)
return[data[0]];
}
...
};
Findingtheconnectededges/verticesbasedonanattributewhenintheedgeitself.Thegoalistopicktheoneedge(outofpotentiallymany)thathasthehighestwhenattributevalue:
UsingTraversalObjects
160
varconfig={
...
expander:function(config,vertex,path){
vardatasource=config.datasource;
//determinealloutgoingedges
varoutEdges=datasource.getOutEdges(vertex);
if(outEdges.length===0){
return[];//returnanemptyarray
}
//sortalloutgoingedgesaccordingto"when"attribute
outEdges.sort(function(l,r){
if(l.when===r.when){
return0;
}
return(l.when<r.when?-1:1);
});
//returnfirstedge(theonewithhighest"when"value)
varedge=outEdges[0];
try{
varv=datasource.getInVertex(edge);
return[{edge:edge,vertex:v}];
}
catch(e){}
return[];
}
...
};
HandlingUniqueness
Graphsmaycontaincycles.Tobeontopofwhathappenswhenatraversalencountersavertexoranedgeithasalreadyvisited,thereareconfigurationoptions.
Thedefaultconfigurationistovisiteveryvertex,regardlessofwhetheritwasalreadyvisitedinthesametraversal.However,edgeswillbydefaultonlybefollowediftheyarenotalreadypresentinthecurrentpath.
Imaginethefollowinggraphwhichcontainsacycle:
A->B->C->A
WhenthetraversalfindstheedgefromCtoA,itwillbydefaultfollowit.Thisisbecausewehavenotseenthisedgeyet.ItwillalsovisitvertexAagain.Thisisbecausebydefaultallverticeswillbevisited,regardlessofwhetheralreadyvisitedornot.
However,thetraversalwillnotagainfollowingtheoutgoingedgefromAtoB.ThisisbecausewealreadyhavetheedgefromAtoBinourcurrentpath.
Thesedefaultsettingswillpreventinfinitetraversals.
Toadjusttheuniquenessforvisitingvertices,therearethefollowingoptionsforuniqueness.vertices:
"none":alwaysvisitavertices,regardlessofwhetheritwasalreadyvisitedornot"global":visitavertexonlyifitwasnotvisitedinthetraversal"path":visitavertexifitisnotincludedinthecurrentpath
Toadjusttheuniquenessforfollowingedges,therearethefollowingoptionsforuniqueness.edges:
"none":alwaysfollowanedge,regardlessofwhetheritwasfollowedbefore"global":followanedgeonlyifitwasn'tfollowedinthetraversal"path":followanedgeifitisnotincludedinthecurrentpath
Notethatuniquenesscheckingwillhavesomeeffectonbothruntimeandmemoryusage.Forexample,whenuniquenesschecksaresetto"global" ,arraysofvisitedverticesandedgesmustbekeptinmemorywhilethetraversalisexecuted.Globaluniquenessshouldthusonlybeusedwhenatraversalisexpectedtovisitfewnodes.
UsingTraversalObjects
161
Intermsofruntime,turningoffuniquenesschecks(bysettingbothoptionsto"none")isthebestchoice,butitisonlysafeforgraphsthatdonotcontaincycles.Whenuniquenesschecksaredeactivatedinagraphwithcycles,thetraversalmightnotabortinasensibleamountoftime.
Optimizations
Thereareafewoptionsformakingatraversalrunfaster.
Thebestoptionistomaketheamountofvisitedverticesandfollowededgesassmallaspossible.Thiscanbeachievedbywritingcustomfilterandexpanderfunctions.Suchfunctionsshouldonlyincludeverticesofinterest,andonlyfollowedgesthatmightbeinteresting.
TraversaldepthcanalsobeboundedwiththeminDepthandmaxDepthoptions.
Anotherwaytospeeduptraversalsistowriteacustomvisitorfunction.Thedefaultvisitorfunction(trackingVisitor)willcopyeveryvisitedvertexintotheresult.Ifverticescontainlotsofdata,thismightbeexpensive.Itisthereforerecommendedtoonlycopysuchdataintotheresultthatisactuallyneeded.Thedefaultvisitorfunctionwillalsocopythefullpathtothevisiteddocumentintotheresult.Thisisevenmoreexpensiveandshouldbeavoidedifpossible.
Ifthegoalofatraversalistoonlycountthenumberofvisitedvertices,theprefabcountingVisitorwillbemuchmoreefficientthanthedefaultvisitor.
Forgraphsthatareknowntonotcontainanycycles,uniquenesschecksshouldbeturnedoff.Thiscanachievedviatheuniquenessconfigurationoptions.Notethatuniquenesschecksshouldnotbeturnedoffforgraphsthatareknowncontaincyclesorifthereisnoinformationaboutthegraph'sstructure.
Bydefault,atraversalwillonlyprocessalimitednumberofvertices.Thisisprotecttheuserfromunintentionallyrunanever-endingtraversalonagraphwithcyclicdata.HowmanyverticeswillbeprocessedatmostisdeterminedbythemaxIterationsconfigurationoption.IfatraversalhitsthecapspecifiedbymaxIterations,itwillabortandthrowatoomanyiterationsexception.Ifthiserrorisencountered,themaxIterationsvalueshouldbeincreasedifitismadesurethattheothertraversalconfigurationparametersaresaneandthetraversalwillabortnaturallyatsomepoint.
Finally,thebuildVerticesconfigurationoptioncanbesettofalsetoavoidlookingupandfullyconstructingvertexdata.Ifallthat'sneededfromverticesarethe_idor_keyattributes,thebuildverticesoptioncanbesettofalse.Ifvisitor,filterorexpandFilterfunctionsneedtoaccessothervertexattributes,theoptionshouldnotbechanged.
ConfigurationOverview
Thissectionsummarizestheconfigurationattributesforthetraversalobject.Theconfigurationcanconsistofthefollowingattributes:
visitor:visitorfunctionforvertices.Itwillbecalledforallnon-excludedvertices.Thegeneralvisitorfunctionsignatureisfunction(config,result,vertex,path).Ifthetraversalorderispreorder-expander,theconnectingedgesofthevisitedvertexwillbepassedasthefifthparameter,extendingthefunctionsignatureto:function(config,result,vertex,path,edges).
Visitorfunctionsarenotexpectedtoreturnvalues,buttheymaymodifytheresultvariableasneeded(e.g.bypushingvertexdataintotheresult).
expander:expanderfunctionthatisresponsibleforreturningedgesandverticesdirectlyconnectedtoavertex.Thefunctionsignatureisfunction(config,vertex,path).Theexpanderfunctionisrequiredtoreturnanarrayofconnectionobjects,consistingofanedgeandvertexattributeeach.Iftherearenoconnectingedges,theexpanderisexpectedtoreturnanemptyarray.filter:vertexfilterfunction.Thefunctionsignatureisfunction(config,vertex,path).Itmayreturnoneofthefollowingvalues:
undefined:vertexwillbeincludedintheresultandconnectededgeswillbetraversed"exclude":vertexwillnotbeincludedintheresultandconnectededgeswillbetraversed"prune":vertexwillbeincludedintheresultbutconnectededgeswillnotbetraversed["prune" ,"exclude" ]:vertexwillnotbeincludedintheresultandconnectededgeswillnotbereturned
expandFilter:filterfunctionappliedoneachedge/vertexcombinationdeterminedbytheexpander.Thefunctionsignatureisfunction(config,vertex,edge,path).Thefunctionshouldreturntrueiftheedge/vertexcombinationshouldbeprocessed,andfalseifitshouldbeignored.sort:afilterfunctiontodeterminetheorderinwhichconnectededgesareprocessed.Thefunctionsignatureisfunction(l,r).Thefunctionisrequiredtoreturnoneofthefollowingvalues:
-1iflshouldhaveasortvaluelessthanr
UsingTraversalObjects
162
1iflshouldhaveahighersortvaluethanr0iflandrhavethesamesortvalue
strategy:determinesthevisitationstrategy.Possiblevaluesaredepthfirstandbreadthfirst.order:determinesthevisitationorder.Possiblevaluesarepreorder,postorder,andpreorder-expander.preorder-expanderisthesameaspreorder,exceptthatthesignatureofthevisitorfunctionwillchangeasdescribedabove.itemOrder:determinestheorderinwhichconnectionsreturnedbytheexpanderwillbeprocessed.Possiblevaluesareforwardandbackward.maxDepth:ifsettoavaluegreaterthan0,thiswilllimitthetraversaltothismaximumdepth.minDepth:ifsettoavaluegreaterthan0,allverticesfoundonalevelbelowtheminDepthlevelwillnotbeincludedintheresult.maxIterations:themaximumnumberofiterationsthatthetraversalisallowedtoperform.Itissensibletosetthisnumbersounboundedtraversalswillterminateatsomepoint.uniqueness:anobjectthatdefineshowrepeatedvisitationsofverticesshouldbehandled.Theuniquenessobjectcanhaveasub-attributevertices,andasub-attributeedges.Eachsub-attributecanhaveoneofthefollowingvalues:
"none":nouniquenessconstraints"path":elementisexcludedifitisalreadycontainedinthecurrentpath.Thissettingmaybesensibleforgraphsthatcontaincycles(e.g.A→B→C→A)."global":elementisexcludedifitwasalreadyfound/visitedatanypointduringthetraversal.
buildVertices:thisattributecontrolswhetherverticesencounteredduringthetraversalwillbelookedupinthedatabaseandwillbemadeavailabletovisitor,filter,andexpandFilterfunctions.Bydefault,verticeswillbelookedupandmadeavailable.However,therearesomespecialusecaseswhenfullyconstructingvertexobjectsisnotnecessaryandcanbeavoided.Forexample,ifatraversalismeanttoonlycountthenumberofvisitedverticesbutdonotreadanydatafromvertices,thisoptionmightbesettotrue.
UsingTraversalObjects
163
ExampleDataThefollowingexamplesalluseavertexcollectionvandanedgecollectione.Thevertexcollectionvcontainscontinents,countries,andcapitals.Theedgecollectionecontainsconnectionsbetweencontinentsandcountries,andbetweencountriesandcapitals.
Tosetupthecollectionsandpopulatethemwithinitialdata,thefollowingscriptwasused:
db._create("v");
db._createEdgeCollection("e");
//vertices:rootnode
db.v.save({_key:"world",name:"World",type:"root"});
//vertices:continents
db.v.save({_key:"continent-africa",name:"Africa",type:"continent"});
db.v.save({_key:"continent-asia",name:"Asia",type:"continent"});
db.v.save({_key:"continent-australia",name:"Australia",type:"continent"});
db.v.save({_key:"continent-europe",name:"Europe",type:"continent"});
db.v.save({_key:"continent-north-america",name:"NorthAmerica",type:"continent"});
db.v.save({_key:"continent-south-america",name:"SouthAmerica",type:"continent"});
//vertices:countries
db.v.save({_key:"country-afghanistan",name:"Afghanistan",type:"country",code:"AFG"});
db.v.save({_key:"country-albania",name:"Albania",type:"country",code:"ALB"});
db.v.save({_key:"country-algeria",name:"Algeria",type:"country",code:"DZA"});
db.v.save({_key:"country-andorra",name:"Andorra",type:"country",code:"AND"});
db.v.save({_key:"country-angola",name:"Angola",type:"country",code:"AGO"});
db.v.save({_key:"country-antigua-and-barbuda",name:"AntiguaandBarbuda",type:"country",code:"ATG"});
db.v.save({_key:"country-argentina",name:"Argentina",type:"country",code:"ARG"});
db.v.save({_key:"country-australia",name:"Australia",type:"country",code:"AUS"});
db.v.save({_key:"country-austria",name:"Austria",type:"country",code:"AUT"});
db.v.save({_key:"country-bahamas",name:"Bahamas",type:"country",code:"BHS"});
db.v.save({_key:"country-bahrain",name:"Bahrain",type:"country",code:"BHR"});
db.v.save({_key:"country-bangladesh",name:"Bangladesh",type:"country",code:"BGD"});
db.v.save({_key:"country-barbados",name:"Barbados",type:"country",code:"BRB"});
db.v.save({_key:"country-belgium",name:"Belgium",type:"country",code:"BEL"});
db.v.save({_key:"country-bhutan",name:"Bhutan",type:"country",code:"BTN"});
db.v.save({_key:"country-bolivia",name:"Bolivia",type:"country",code:"BOL"});
db.v.save({_key:"country-bosnia-and-herzegovina",name:"BosniaandHerzegovina",type:"country",code:"BIH"});
db.v.save({_key:"country-botswana",name:"Botswana",type:"country",code:"BWA"});
db.v.save({_key:"country-brazil",name:"Brazil",type:"country",code:"BRA"});
db.v.save({_key:"country-brunei",name:"Brunei",type:"country",code:"BRN"});
db.v.save({_key:"country-bulgaria",name:"Bulgaria",type:"country",code:"BGR"});
db.v.save({_key:"country-burkina-faso",name:"BurkinaFaso",type:"country",code:"BFA"});
db.v.save({_key:"country-burundi",name:"Burundi",type:"country",code:"BDI"});
db.v.save({_key:"country-cambodia",name:"Cambodia",type:"country",code:"KHM"});
db.v.save({_key:"country-cameroon",name:"Cameroon",type:"country",code:"CMR"});
db.v.save({_key:"country-canada",name:"Canada",type:"country",code:"CAN"});
db.v.save({_key:"country-chad",name:"Chad",type:"country",code:"TCD"});
db.v.save({_key:"country-chile",name:"Chile",type:"country",code:"CHL"});
db.v.save({_key:"country-colombia",name:"Colombia",type:"country",code:"COL"});
db.v.save({_key:"country-cote-d-ivoire",name:"Coted'Ivoire",type:"country",code:"CIV"});
db.v.save({_key:"country-croatia",name:"Croatia",type:"country",code:"HRV"});
db.v.save({_key:"country-czech-republic",name:"CzechRepublic",type:"country",code:"CZE"});
db.v.save({_key:"country-denmark",name:"Denmark",type:"country",code:"DNK"});
db.v.save({_key:"country-ecuador",name:"Ecuador",type:"country",code:"ECU"});
db.v.save({_key:"country-egypt",name:"Egypt",type:"country",code:"EGY"});
db.v.save({_key:"country-eritrea",name:"Eritrea",type:"country",code:"ERI"});
db.v.save({_key:"country-finland",name:"Finland",type:"country",code:"FIN"});
db.v.save({_key:"country-france",name:"France",type:"country",code:"FRA"});
db.v.save({_key:"country-germany",name:"Germany",type:"country",code:"DEU"});
db.v.save({_key:"country-people-s-republic-of-china",name:"People'sRepublicofChina",type:"country",code:"CHN"});
//vertices:capitals
db.v.save({_key:"capital-algiers",name:"Algiers",type:"capital"});
db.v.save({_key:"capital-andorra-la-vella",name:"AndorralaVella",type:"capital"});
db.v.save({_key:"capital-asmara",name:"Asmara",type:"capital"});
db.v.save({_key:"capital-bandar-seri-begawan",name:"BandarSeriBegawan",type:"capital"});
db.v.save({_key:"capital-beijing",name:"Beijing",type:"capital"});
db.v.save({_key:"capital-berlin",name:"Berlin",type:"capital"});
db.v.save({_key:"capital-bogota",name:"Bogota",type:"capital"});
db.v.save({_key:"capital-brasilia",name:"Brasilia",type:"capital"});
ExampleData
164
db.v.save({_key:"capital-bridgetown",name:"Bridgetown",type:"capital"});
db.v.save({_key:"capital-brussels",name:"Brussels",type:"capital"});
db.v.save({_key:"capital-buenos-aires",name:"BuenosAires",type:"capital"});
db.v.save({_key:"capital-bujumbura",name:"Bujumbura",type:"capital"});
db.v.save({_key:"capital-cairo",name:"Cairo",type:"capital"});
db.v.save({_key:"capital-canberra",name:"Canberra",type:"capital"});
db.v.save({_key:"capital-copenhagen",name:"Copenhagen",type:"capital"});
db.v.save({_key:"capital-dhaka",name:"Dhaka",type:"capital"});
db.v.save({_key:"capital-gaborone",name:"Gaborone",type:"capital"});
db.v.save({_key:"capital-helsinki",name:"Helsinki",type:"capital"});
db.v.save({_key:"capital-kabul",name:"Kabul",type:"capital"});
db.v.save({_key:"capital-la-paz",name:"LaPaz",type:"capital"});
db.v.save({_key:"capital-luanda",name:"Luanda",type:"capital"});
db.v.save({_key:"capital-manama",name:"Manama",type:"capital"});
db.v.save({_key:"capital-nassau",name:"Nassau",type:"capital"});
db.v.save({_key:"capital-n-djamena",name:"N'Djamena",type:"capital"});
db.v.save({_key:"capital-ottawa",name:"Ottawa",type:"capital"});
db.v.save({_key:"capital-ouagadougou",name:"Ouagadougou",type:"capital"});
db.v.save({_key:"capital-paris",name:"Paris",type:"capital"});
db.v.save({_key:"capital-phnom-penh",name:"PhnomPenh",type:"capital"});
db.v.save({_key:"capital-prague",name:"Prague",type:"capital"});
db.v.save({_key:"capital-quito",name:"Quito",type:"capital"});
db.v.save({_key:"capital-saint-john-s",name:"SaintJohn's",type:"capital"});
db.v.save({_key:"capital-santiago",name:"Santiago",type:"capital"});
db.v.save({_key:"capital-sarajevo",name:"Sarajevo",type:"capital"});
db.v.save({_key:"capital-sofia",name:"Sofia",type:"capital"});
db.v.save({_key:"capital-thimphu",name:"Thimphu",type:"capital"});
db.v.save({_key:"capital-tirana",name:"Tirana",type:"capital"});
db.v.save({_key:"capital-vienna",name:"Vienna",type:"capital"});
db.v.save({_key:"capital-yamoussoukro",name:"Yamoussoukro",type:"capital"});
db.v.save({_key:"capital-yaounde",name:"Yaounde",type:"capital"});
db.v.save({_key:"capital-zagreb",name:"Zagreb",type:"capital"});
//edges:continent->world
db.e.save("v/continent-africa","v/world",{type:"is-in"});
db.e.save("v/continent-asia","v/world",{type:"is-in"});
db.e.save("v/continent-australia","v/world",{type:"is-in"});
db.e.save("v/continent-europe","v/world",{type:"is-in"});
db.e.save("v/continent-north-america","v/world",{type:"is-in"});
db.e.save("v/continent-south-america","v/world",{type:"is-in"});
//edges:country->continent
db.e.save("v/country-afghanistan","v/continent-asia",{type:"is-in"});
db.e.save("v/country-albania","v/continent-europe",{type:"is-in"});
db.e.save("v/country-algeria","v/continent-africa",{type:"is-in"});
db.e.save("v/country-andorra","v/continent-europe",{type:"is-in"});
db.e.save("v/country-angola","v/continent-africa",{type:"is-in"});
db.e.save("v/country-antigua-and-barbuda","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-argentina","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-australia","v/continent-australia",{type:"is-in"});
db.e.save("v/country-austria","v/continent-europe",{type:"is-in"});
db.e.save("v/country-bahamas","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-bahrain","v/continent-asia",{type:"is-in"});
db.e.save("v/country-bangladesh","v/continent-asia",{type:"is-in"});
db.e.save("v/country-barbados","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-belgium","v/continent-europe",{type:"is-in"});
db.e.save("v/country-bhutan","v/continent-asia",{type:"is-in"});
db.e.save("v/country-bolivia","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-bosnia-and-herzegovina","v/continent-europe",{type:"is-in"});
db.e.save("v/country-botswana","v/continent-africa",{type:"is-in"});
db.e.save("v/country-brazil","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-brunei","v/continent-asia",{type:"is-in"});
db.e.save("v/country-bulgaria","v/continent-europe",{type:"is-in"});
db.e.save("v/country-burkina-faso","v/continent-africa",{type:"is-in"});
db.e.save("v/country-burundi","v/continent-africa",{type:"is-in"});
db.e.save("v/country-cambodia","v/continent-asia",{type:"is-in"});
db.e.save("v/country-cameroon","v/continent-africa",{type:"is-in"});
db.e.save("v/country-canada","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-chad","v/continent-africa",{type:"is-in"});
db.e.save("v/country-chile","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-colombia","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-cote-d-ivoire","v/continent-africa",{type:"is-in"});
db.e.save("v/country-croatia","v/continent-europe",{type:"is-in"});
db.e.save("v/country-czech-republic","v/continent-europe",{type:"is-in"});
db.e.save("v/country-denmark","v/continent-europe",{type:"is-in"});
db.e.save("v/country-ecuador","v/continent-south-america",{type:"is-in"});
ExampleData
165
db.e.save("v/country-egypt","v/continent-africa",{type:"is-in"});
db.e.save("v/country-eritrea","v/continent-africa",{type:"is-in"});
db.e.save("v/country-finland","v/continent-europe",{type:"is-in"});
db.e.save("v/country-france","v/continent-europe",{type:"is-in"});
db.e.save("v/country-germany","v/continent-europe",{type:"is-in"});
db.e.save("v/country-people-s-republic-of-china","v/continent-asia",{type:"is-in"});
//edges:capital->country
db.e.save("v/capital-algiers","v/country-algeria",{type:"is-in"});
db.e.save("v/capital-andorra-la-vella","v/country-andorra",{type:"is-in"});
db.e.save("v/capital-asmara","v/country-eritrea",{type:"is-in"});
db.e.save("v/capital-bandar-seri-begawan","v/country-brunei",{type:"is-in"});
db.e.save("v/capital-beijing","v/country-people-s-republic-of-china",{type:"is-in"});
db.e.save("v/capital-berlin","v/country-germany",{type:"is-in"});
db.e.save("v/capital-bogota","v/country-colombia",{type:"is-in"});
db.e.save("v/capital-brasilia","v/country-brazil",{type:"is-in"});
db.e.save("v/capital-bridgetown","v/country-barbados",{type:"is-in"});
db.e.save("v/capital-brussels","v/country-belgium",{type:"is-in"});
db.e.save("v/capital-buenos-aires","v/country-argentina",{type:"is-in"});
db.e.save("v/capital-bujumbura","v/country-burundi",{type:"is-in"});
db.e.save("v/capital-cairo","v/country-egypt",{type:"is-in"});
db.e.save("v/capital-canberra","v/country-australia",{type:"is-in"});
db.e.save("v/capital-copenhagen","v/country-denmark",{type:"is-in"});
db.e.save("v/capital-dhaka","v/country-bangladesh",{type:"is-in"});
db.e.save("v/capital-gaborone","v/country-botswana",{type:"is-in"});
db.e.save("v/capital-helsinki","v/country-finland",{type:"is-in"});
db.e.save("v/capital-kabul","v/country-afghanistan",{type:"is-in"});
db.e.save("v/capital-la-paz","v/country-bolivia",{type:"is-in"});
db.e.save("v/capital-luanda","v/country-angola",{type:"is-in"});
db.e.save("v/capital-manama","v/country-bahrain",{type:"is-in"});
db.e.save("v/capital-nassau","v/country-bahamas",{type:"is-in"});
db.e.save("v/capital-n-djamena","v/country-chad",{type:"is-in"});
db.e.save("v/capital-ottawa","v/country-canada",{type:"is-in"});
db.e.save("v/capital-ouagadougou","v/country-burkina-faso",{type:"is-in"});
db.e.save("v/capital-paris","v/country-france",{type:"is-in"});
db.e.save("v/capital-phnom-penh","v/country-cambodia",{type:"is-in"});
db.e.save("v/capital-prague","v/country-czech-republic",{type:"is-in"});
db.e.save("v/capital-quito","v/country-ecuador",{type:"is-in"});
db.e.save("v/capital-saint-john-s","v/country-antigua-and-barbuda",{type:"is-in"});
db.e.save("v/capital-santiago","v/country-chile",{type:"is-in"});
db.e.save("v/capital-sarajevo","v/country-bosnia-and-herzegovina",{type:"is-in"});
db.e.save("v/capital-sofia","v/country-bulgaria",{type:"is-in"});
db.e.save("v/capital-thimphu","v/country-bhutan",{type:"is-in"});
db.e.save("v/capital-tirana","v/country-albania",{type:"is-in"});
db.e.save("v/capital-vienna","v/country-austria",{type:"is-in"});
db.e.save("v/capital-yamoussoukro","v/country-cote-d-ivoire",{type:"is-in"});
db.e.save("v/capital-yaounde","v/country-cameroon",{type:"is-in"});
db.e.save("v/capital-zagreb","v/country-croatia",{type:"is-in"});
ExampleData
166
Edges,Identifiers,HandlesThisisanintroductiontoArangoDB'sinterfaceforedges.Edgesmaybeusedingraphs.HereweworkwithedgesfromtheJavaScriptshellarangosh.ForotherlanguagesseethecorrespondinglanguageAPI.
Agraphdatamodelalwaysconsistsofatleasttwocollections:therelationsbetweenthenodesinthegraphsarestoredinan"edgescollection",thenodesinthegrapharestoredindocumentsinregularcollections.
EdgesinArangoDBarespecialdocuments.Inadditiontothesystemattributes_key,_idand_rev,theyhavetheattributes_fromand_to,whichcontaindocumenthandles,namelythestart-pointandtheend-pointoftheedge.
Example:
the"edge"collectionstorestheinformationthatacompany'sreceptionissub-unittotheservicesunitandtheservicesunitissub-unittotheCEO.Youwouldexpressthisrelationshipwiththe_fromand_toattributesthe"normal"collectionstoresallthepropertiesaboutthereception,e.g.that20peopleareworkingthereandtheroomnumberetc_fromisthedocumenthandleofthelinkedvertex(incomingrelation)_toisthedocumenthandleofthelinkedvertex(outgoingrelation)
Edgecollectionsarespecialcollectionsthatstoreedgedocuments.Edgedocumentsareconnectiondocumentsthatreferenceotherdocuments.Thetypeofacollectionmustbespecifiedwhenacollectioniscreatedandcannotbechangedafterwards.
Tochangeedgeendpointsyouwouldneedtoremoveolddocument/edgeandinsertnewone.Otherfieldscanbeupdatedasindefaultcollection.
WorkingwithEdges
Edgesarenormaldocumentsthatalwayscontaina_fromanda_toattribute.
WorkingwithEdges
167
DistributedIterativeGraphProcessing(Pregel)Distributedgraphprocessingenablesyoutodoonlineanalyticalprocessingdirectlyongraphsstoredintoarangodb.Thisisintendedtohelpyougainanalyticalinsightsonyourdata,withouthavingtouseexternalprocessingsytems.ExamplesofalgorithmstoexecutearePageRank,VertexCentrality,VertexCloseness,ConnectedComponents,CommunityDetection.Thissystemisnotusefulfortypicalonlinequeries,whereyoujustdoworkonasmallsetofvertices.ThesekindoftasksarebettersuitedforAQL.
TheprocessingsysteminsideArangoDBisbasedon:Pregel:ASystemforLarge-ScaleGraphProcessing–Malewiczetal.(Google)2010Thisconceptenablesustoperformdistributedgraphprocessing,withouttheneedfordistributedgloballocking.
PrerequisitesIfyouarerunningasingleArangoDBinstanceinsingle-servermode,therearenorequirementsregardingthemodelingofyourdata.Allyouneedisatleastonevertexcollectionandoneedgecollection.Notethattheperformancemaybebetter,ifthenumberofyourshards/collectionsmatchesthenumberofCPUcores.
WhenyouuseArangoDBCommunityeditioninclustermode,youmightneedtomodelyourcollectionsinacertainwaytoensurecorrectresults.Formoreinformationseethenextsection.
RequirementsforCollectionsinaCluster(NonSmartGraph)Toenableiterativegraphprocessingforyourdata,youwillneedtoensurethatyourvertexandedgecollectionsareshardedinaspecificway.
ThepregelcomputingmodelrequiresalledgestobepresentontheDBServerwherethevertexdocumentidentifiedbythe_fromvalueislocated.Thismeansthevertexcollectionsneedtobeshardedby'_key'andtheedgecollectionwillneedtobeshardedafteranattributewhichalwayscontainsthe'_key'ofthevertex.
Ourimplementationcurrentlyrequireseveryedgecollectiontobeshardedaftera"vertex"attributes,additionallyyouwillneedtospecifythekeydistributeShardsLikeandanequalnumberofshardsoneverycollection.OnlyiftheserequirementsaremetcanArangoDBplacetheedgesandverticescorrectly.
Forexampleyoumightcreateyourcollectionslikethis:
//Createmainvertexcollection:
db._create("vertices",{
shardKeys:['_key'],
numberOfShards:8
});
//Optionallycreatearbitraryadditionalvertexcollections
db._create("additonal",{
distributeShardsLike:"vertices",
numberOfShards:8
});
//Create(oneormore)edge-collections:
db._createEdgeCollection("edges",{
shardKeys:['vertex'],
distributeShardsLike:"vertices",
numberOfShards:8
});
Youwillneedtoensurethatedgedocumentscontainthepropervaluesintheirshardingattribute.Foravertexdocumentwiththefollowingcontent{_key:"A",value:0}thecorrespondingedgedocumentswouldhavelooklikethis:
{_from:"vertices/A",_to:"vertices/B",vertex:"A"}
{_from:"vertices/A",_to:"vertices/C",vertex:"A"}
{_from:"vertices/A",_to:"vertices/D",vertex:"A"}
...
Pregel
168
ThiswillensurethatoutgoingedgedocumentswillbeplacedonthesameDBServerasthevertex.Withoutthecorrectplacementoftheedges,thepregelgraphprocessingsystemwillnotworkcorrectly,becauseedgeswillnotloadcorrectly.
ArangoshAPI
StartinganAlgorithmExecution
ThepregelAPIisaccessiblethroughthe@arangodb/pregelpackage.Tostartanexecutionyouneedtospecifythealgorithmnameandthevertexandedgecollections.Alternativelyyoucanspecifyanamedgraph.Additionallyyoucanspecifycustomparameterswhichvaryforeachalgorithm.ThestartmethodwillalwaysauniqueIDwhichcanbeusedtointeractwiththealgorithmandlateron.
Thebelowversionofthestartmethodcanbeusedfornamedgraphs:
varpregel=require("@arangodb/pregel");
varparams={};
varexecution=pregel.start("<algorithm>","<yourgraph>",params);
Paramsneedstobeanobject,thevalidkeysarementionedbelowinthesectionAlgorithms
Alternativelyyoumightwanttospecifythevertexandedgecollectionsdirectly.Thecall-syntaxofthestart``methodchangesinthiscase.ThesecondargumentmustbeanobjectwiththekeysvertexCollectionsandedgeCollections`.
varpregel=require("@arangodb/pregel");
varparams={};
varexecution=pregel.start("<algorithm>",{vertexCollections:["vertices"],edgeCollections:["edges"]},{});
Thelastargumentisstilltheparameterobject.Seebelowforalistofalgorithmsandparameters.
StatusofanAlgorithmExecution
Thecodereturnedbythepregel.start(...)methodcanbeusedtotrackthestatusofyouralgorithm.
varexecution=pregel.start("sssp","demograph",{source:"vertices/V"});
varstatus=pregel.status(execution);
Theresultwilltellyouthecurrentstatusofthealgorithmexecution.Itwilltellyouthecurrentstateoftheexecution,thecurrentglobalsuperstep,theruntime,theglobalaggregatorvaluesaswellasthenumberofsendandreceivedmessages.
Validvaluesforthestatefieldinclude:
"running"algorithmisstillrunning"done":Theexecutionisdone,theresultmightnotbewrittenbackintothecollectionyet."canceled":Theexecutionwaspermanentlycanceled,eitherbytheuserorbyanerror."inerror":Theexeuctionisinanerrorstate.ThiscanbecausedbyprimaryDBServersbeingnotreachableorbeingnonresponsive.Theexecutionmightrecoverlater,orswitchto"canceled"ifitwasnotabletorecoversuccessfuly"recovering":Theexecutionisactivelyrecovering,willswitchbackto"running"iftherecoverywassuccessful
Theobjectreturnedbythestatusmethodmightforexamplelooksomethinglikethis:
{
"state":"running",
"gss":12,
"totalRuntime":123.23,
"aggregators":{
"converged":false,
"max":true,
"phase":2
},
"sendCount":3240364978,
"receivedCount":3240364975
}
Pregel
169
CancelinganExecution/Discardingresults
Tocancelanexecutionwhichisstillrunnning,anddiscardanyintermediareresultsyoucanusethecancelmethod.Thiswillimmediatlyfreeallmemorytakenupbytheexecution,andwillmakeyouloseallintermediarydata.
Youmightgetinconsistentresultsifyoucancelanexecutionwhileitisalreadyinit'sdonestate.Thedataiswrittenmulti-threadedintoallcollectionshardsatonce,thismeanstherearemultipletransactionssimultaniously.Atransactionmightalreadybecommitedwhenyoucanceltheexecutionjob,thereforeyoumightseetheresultinyourcollection.Thisdoesnotapplyifyouconfiguredtheexecutiontonotwritedataintothecollection.
//startasinglesourceshortestpathjob
varexecution=pregel.start("sssp","demograph",{source:"vertices/V"});
pregel.cancel(execution);
AQLintegrationArangoDBsupportsretrievingtemporarypregelresultsthroughtheArangoDBquerylanguage(AQL).Whenourgraphprocessingsubsystemfinishesexecutinganalgorithm,theresultcaneitherbewrittenbackintothedatabaseorkeptinmemory.InbothcasestheresultcanbequeriedviaAQL.Ifthedatawasnotwrittentothedatabasestoreitisonlyheldtemporarily,untiltheusercallsthecancelmethodForexampleausermightwanttoqueryonlynodeswiththemostrankfromtheresultsetofaPageRankexecution.
FORvINPREGEL_RESULT(<handle>)
FILTERv.value>=0.01
RETURNv._key
AvailableAlgorithmsThereareanumberofgeneralparameterswhichapplytoalmostallalgorithms:
store:Isperdefaulttrue,thepregelenginewillwriteresultsbacktothedatabase.ifthevalueisfalsethenyoucanquerytheresultsviaAQLk,seeAQLintegration.maxGSS:Maximumnumberofglobaliterationsforthisalgorithmparallelism:Numberofparellelthreadstouseperworker.Doesnotinfluencethenumberofthreadsusedtoload
orstoredatafromthedatabase(thisdependsonthenumberofshards).
async:Algorithmswichsupportasyncmode,willrunwithoutsynchronizedglobaliterations,mightleadtoperformanceincreasesifyouhaveloadimbalances.resultField:Mostalgorithmswillwritetheresultintothisfield
PageRank
PageRankisawellknownalgorithmtorankdocumentsinagraph.Thealgorithmwillrununtiltheexecutionconverges.Specifyacustomthresholdwiththeparameterthreshold,torunforafixednumberofiterationsusethemaxGSSparameter.
varpregel=require("@arangodb/pregel");
pregel.start("pagerank","graphname",{maxGSS:100,threshold:0.00000001})
Single-SourceShortestPath
Calculatesthedistanceofeachvertextoacertainshortestpath.Thealgorithmwillrununtilitconverges,theiterationsareboundbythediameter(thelongestshortestpath)ofyourgraph.
varpregel=require("@arangodb/pregel");
pregel.start("sssp","graphname",{source:"vertices/1337"})
ConnectedComponents
Pregel
170
Therearetwoalgorithmstofindconnectedcomponentsinagraph.Tofindweaklyconnectedcomponents(WCC)youcanusethealgorithmnamed"connectedcomponents",tofindstronglyconnectedcomponents(SCC)youcanusethealgorithmnamed"scc".BothalgorithmwillassignacomponentIDtoeachvertex.
Aweaklyconnectedcomponentsmeansthatthereexistapathfromeveryvertexpairinthatcomponent.WCCisaverysimpleandfastalgorithm,whichwillonlyworkcorrectlyonundirectedgraphs.Yourresultsondirectedgraphsmayvary,dependingonhowconnectedyourcomponentsare.
InthecaseofSCCacomponentmeanseveryvertexisreachablefromanyothervertexinthesamecomponent.ThealgorithmismorecomplexthantheWCCalgorithmandrequiresmoreRAM,becauseeachvertexneedstostoremuchmorestate.ConsiderusingWCCifyouthinkyourdatamaybesuitableforit.
varpregel=require("@arangodb/pregel");
//weaklyconnectedcomponents
pregel.start("connectedcomponents","graphname")
//stronglyconnectedcomponents
pregel.start("scc","graphname")
Hyperlink-InducedTopicSearch(HITS)
HITSisalinkanalysisalgorithmthatratesWebpages,developedbyJonKleinberg(Thealgorithmisalsoknownashubsandauthorities).
TheideabehindHubsandAuthoritiescomesfromthetypicalstructureoftheweb:Certainwebsitesknownashubs,serveaslargedirectoriesthatarenotactuallyauthoritativeontheinformationthattheyhold.Thesehubsareusedascompilationsofabroadcatalogofinformationthatleadsusersdirecttootherauthoritativewebpages.Thealgorithmassignseachvertextwoscores:Theauthority-scoreandthehub-score.Theauthorityscorerateshowmanygoodhubspointtoaparticularvertex(orwebpage),thehubscorerateshowgood(authoritative)theverticespointedtoare.Formoreseehttps://en.wikipedia.org/wiki/HITS_algorithm
Ourversionofthealgorithmconvergesafteracertainamountoftime.Theparameterthresholdcanbeusedtosetalimitfortheconvergence(measuredasmaximumabsolutedifferenceofthehubandauthorityscoresbetweenthecurrentandlastiteration)Whenyouspecifytheresultfieldname,thehubscorewillbestoredin"_hub"andtheauthorityscorein"_auth".Thealgorithmcanbeexecutedlikethis:
varpregel=require("@arangodb/pregel");
varhandle=pregel.start("hits","yourgraph",{threshold:0.00001,resultField:"score"});
VertexCentrality
Centralitymeasureshelpidentifythemostimportantverticesinagraph.Theycanbeusedinawiderangeofapplications:Forexampletheycanbeusedtoidentifyinfluencersinsocialnetworks,ormiddle-meninterroristnetworks.Therearevariousdefinitionsforcentrality,thesimplestonebeingthevertexdegree.Thesedefinitionswerenotdesignedwithscalabilityinmind.Itisprobablyimpossibletodiscoveranefficientalgorithmwhichcomputestheminadistributedway.Fortunatelytherearescalablesubstitutionsavailable,whichshouldbeequallyusableformostusecases.
Pregel
171
EffectiveCloseness
Acommondefinitionsofcentralityistheclosenesscentrality(orcloseness).Theclosenessofavertexinagraphistheinverseaveragelengthoftheshortestpathbetweenthevertexandallothervertices.Forverticesx,yandshortestdistanced(y,x)itisdefinedas
EffectiveClosenessapproximatestheclosenessmeasure.Thealgorithmworksbyiterativelyestimatingthenumberofshortestpathspassingthrougheachvertex.Thescorewillapproximatesthetherealclosenessscore,sinceitisnotpossibletoactuallycountallshortestpathsduetothehorrendousO(n^2d)memoryrequirements.ThealgorithmisfromthepaperCentralitiesinLargeNetworks:AlgorithmsandObservations(UKanget.al.2011)*
ArangoDBsimplementationapproximatesthenumberofshortestpathineachiterationbyusingaHyperLogLogcounterwith64buckets.Thisshouldworkwellonlargegraphsandonsmalleronesaswell.ThememoryrequirementsshouldbeO(n*d)wherenisthenumberofverticesanddthediameterofyourgraph.Eachvertexwillstoreacounterforeachiterationofthealgorithm.Thealgorithmcanbeusedlikethis
constpregel=require("@arangodb/pregel");
consthandle=pregel.start("effectivecloseness","yourgraph",{resultField:"closeness"});
LineRank
Anothercommonmeasureisthebetweenness*centrality:Itmeasuresthenumberoftimesavertexispartofshortestpathsbetweenanypairsofvertices.Foravertexvbetweennessisdefinedas
Wheretheσrepresentsthenumberofshortestpathsbetweenxandy,andσ(v)representsthenumberofpathsalsopassingthroughavertexv.Byintuitionavertexwithhigherbetweenesscentralitywillhavemoreinformationpassingthroughit.
LineRankapproximatestherandomwalkbetweennessofeveryvertexinagraph.Thisistheprobabilitythatsomeonestartingonanarbitaryvertex,willvisitthisnodewhenherandomlychoosesedgestovisit.Thealgoruthmessentiallybuildsalinegraphoutofyourgraph(switchestheverticesandedges),andthencomputesascoresimilartoPageRank.Thiscanbeconsideredascalableequivalenttovertexbetweeness,whichcanbeexecuteddistributedlyinArangoDB.ThealgorithmisfromthepaperCentralitiesinLargeNetworks:AlgorithmsandObservations(UKanget.al.2011)
Pregel
172
constpregel=require("@arangodb/pregel");
consthandle=pregel.start("linerank","yourgraph",{"resultField":"rank"});
CommunityDetection
Graphsbasedonrealworldnetworksoftenhaveacommunitystructure.Thismeansitispossibletofindgroupsofverticessuchthateacheachvertexgroupisinternallymoredenselyconnectedthanoutsidethegroup.Thishasmanyapplicationswhenyouwanttoanalyzeyournetworks,forexampleSocialnetworksincludecommunitygroups(theoriginoftheterm,infact)basedoncommonlocation,interests,occupation,etc.
LabelPropagation
LabelPropagationcanbeusedtoimplementcommunitydetectiononlargegraphs.Theideaisthateachvertexshouldbeinthecommunitythatmostofhisneighboursarein.WeiterativelydeteminethisbyfirstassigningrandomCommunityID's.Theneachitertation,avertexwillsendit'scurrentcommunityIDtoallhisneighborvertices.TheneachvertexadoptsthecommunityIDhereceivedmostfrequentlyduringtheiteration.
Thealgorithmrunsuntilitconverges,whichlikelyneverreallyhappensonlargegraphs.Thereforeyouneedtospecifyamaximumiterationboundwhichsuitsyou.Thedefaultboundis500iterations,whichislikelytoolargeforyourapplication.Shouldworkbestonundirectedgraphs,resultsondirectedgraphsmightvarydependingonthedensityofyourgraph.
constpregel=require("@arangodb/pregel");
consthandle=pregel.start("labelpropagation","yourgraph",{maxGSS:100,resultField:"community"});
Speaker-ListenerLabelPropagation
TheSpeaker-listenerLabelPropagation(SLPA)canbeusedtoimplementcommunitydetection.Itworkssimilartothelabelpropagationalgorithm,butnoweverynodeadditionallyaccumulatesamemoryofobservedlabels(insteadofforgettingallbutonelabel).
Beforethealgorithmrun,everyvertexisinitializedwithanuniqueID(theinitialcommunitylabel).Duringtherunthreestepsareexecutedforeachvertex:
1. Currentvertexisthelistenerallotherverticesarespeakers2. Eachspeakersendsoutalabelfrommemory,wesendoutarandomlabelwithaprobabilityproportionaltothenumberoftimesthe
vertexobservedthelabel3. Thelistenerremembersoneofthelabels,wealwayschoosethemostfrequentlyobservedlabel
constpregel=require("@arangodb/pregel");
consthandle=pregel.start("slpa","yourgraph",{maxGSS:100,resultField:"community"});
YoucanalsoexecuteSLPAwiththemaxCommunitiesparametertolimitthenumberofouputcommunities.Internallythealgorithmwillstillkeepthememoryofalllabels,buttheoutputisreducedtojusthenmostfrequentlyobservedlabels.
constpregel=require("@arangodb/pregel");
consthandle=pregel.start("slpa","yourgraph",{maxGSS:100,resultField:"community",maxCommunities:1});
//checkthestatusperiodicallyforcompletion
pregel.status(handle);
Pregel
173
FoxxTraditionally,server-sideprojectshavebeendevelopedasstandaloneapplicationsthatguidethecommunicationbetweentheclient-sidefrontendandthedatabasebackend.Thishasledtoapplicationsthatwereeitherdevelopedassinglemonolithsorthatduplicateddataaccessanddomainlogicacrossallservicesthathadtoaccessthedatabase.Additionally,toolstoabstractawaytheunderlyingdatabasecallscouldincuralotofnetworkoverheadwhenusingremotedatabaseswithoutcarefuloptimization.
ArangoDBallowsapplicationdeveloperstowritetheirdataaccessanddomainlogicasmicroservicesrunningdirectlywithinthedatabasewithnativeaccesstoin-memorydata.TheFoxxmicroserviceframeworkmakesiteasytoextendArangoDB'sownRESTAPIwithcustomHTTPendpointsusingmodernJavaScriptrunningonthesameV8engineyouknowfromNode.jsandtheGoogleChromewebbrowser.
Unliketraditionalapproachestostoringlogicinthedatabase(likestoredprocedures),thesemicroservicescanbewrittenasregularstructuredJavaScriptapplicationsthatcanbeeasilydistributedandversioncontrolled.Dependingonyourproject'sneedsFoxxcanbeusedtobuildanythingfromoptimizedRESTendpointsperformingcomplexdataaccesstoentirestandaloneapplicationsrunningdirectlyinsidethedatabase.
FoxxMicroservices
174
FoxxataglanceEachFoxxserviceisdefinedbyaJSONmanifestspecifyingtheentrypoint,anyscriptsdefinedbytheservice,possibleconfigurationoptionsandFoxxdependencies,aswellasothermetadata.Withinaservice,theseoptionsareexposedastheservicecontext.
AttheheartoftheFoxxframeworkliestheFoxxRouterwhichisusedtodefineHTTPendpoints.AservicecanaccessthedatabaseeitherdirectlyfromitscontextusingprefixedcollectionsortheArangoDBdatabaseAPI.
WhileFoxxisprimarilydesignedtobeusedtoaccessthedatabaseitself,ArangoDBalsoprovidesanAPItomakeHTTPrequeststoexternalservices.
Scriptscanbeusedtoperformone-offtasks,whichcanalsobescheduledtobeperformedasynchronouslyusingthebuilt-injobqueue.
Finally,FoxxservicescanbeinstalledandmanagedovertheWeb-UIorthroughArangoDBsHTTPAPI.
HowdoesitworkFoxxservicesconsistofJavaScriptcoderunningintheV8JavaScriptruntimeembeddedinsideArangoDB.EachserviceismountedineachavailableV8context(thenumberofcontextscanbeadjustedintheArangoDBconfiguration).Incomingrequestsaredistributedaccrossthesecontextsautomatically.
Ifyou'recomingfromanotherJavaScriptenvironmentlikeNode.jsthisissimilartorunningmultipleNode.jsprocessesbehindaloadbalancer:youshouldnotrelyonserver-sidestate(otherthanthedatabaseitself)betweendifferentrequestsasthereisnowayofmakingsureconsecutiverequestswillbehandledinthesamecontext.
BecausetheJavaScriptcodeisrunninginsidethedatabaseanotherdifferenceisthatallFoxxandArangoDBAPIsarepurelysynchronousandshouldbeconsideredblocking.Thisisespeciallyimportantfortransactions,whichinArangoDBcanexecutearbitrarycodebutmayhavetolockentirecollections(effectivelypreventinganydatatobewritten)untilthecodehascompleted.
Forinformationonhowthisaffectsinteroperabilitywiththird-partyJavaScriptmoduleswrittenforotherJavaScriptenvironmentsseethechapterondependencies.
DevelopmentmodeDevelopmentmodeallowsyoutomakechangestodeployedservicesin-placedirectlyonthedatabaseserver'sfilesystemwithoutdownloadingandre-uploadingtheservicebundle.Additionallyerrormessageswillcontainstacktraces.
YoucantoggledevelopmentmodeonandoffintheservicesettingstabofthewebinterfaceorusingtheHTTPAPI.Onceactivatedtheservice'sfilesystempathwillbeshownintheinfotab.
Onceenabledtheservice'ssourcefilesandmanifestwillbere-evaluated,andthesetupscript(ifpresent)re-executed,everytimearouteoftheserviceisaccessed,effectivelyre-deployingtheserviceoneveryrequest.Asthenameindicatesthisisintendedtobeusedstrictlyduringdevelopmentandismostdefinitelyabadideaonproductionservers.Theadditionalinformationexposedduringdevelopmentmodemayincludefilesystempathsandpartsoftheservice'ssourcecode.
Alsonotethatifyouareservingstaticfilesaspartofyourservice,accessingthesefilesfromabrowsermayalsotriggerare-deploymentoftheservice.Finally,makingHTTPrequeststoaservicerunningindevelopmentmodefromwithintheservice(i.e.usingthe@arangodb/requestmoduletoaccesstheserviceitself)isprobablynotagoodideaeither.
Bewareofdeletingthedatabasetheserviceisdeployedon:itwillerasethesourcefilesoftheservicealongwiththecollections.Youshouldbackupthecodeyouworkedonindevelopmentbeforedoingthattoavoidlosingyourprogress.
FoxxstoreTheFoxxstoreprovidesaccesstoanumberofready-to-useofficialandcommunity-maintainedFoxxservicesyoucaninstallwithasingleclick,includingexampleservicesandwrappersforexternalSaaStoolsliketransactionale-mailservices,bugloggersoranalyticstrackers.
YoucanfindtheFoxxstoreinthewebinterfacebyusingtheAddServicebuttonintheservicelist.
Ataglance
175
Cluster-Foxx
WhenrunningArangoDBinaclustertheFoxxserviceswillrunoneachcoordinator.Installing,upgradinganduninstallingservicesonanycoordinatorwillautomaticallydistributetheservicetotheothercoordinators,makingdeploymentsaseasyasinsingle-servermode.However,thismeanstherearesomelimitations:
Youshouldavoidanykindoffilesystemstatebeyondthedeployedservicebundleitself.Don'twritedatatothefilesystemorencodeanyexpectationsofthefilesystemstateotherthanthefilesintheservicefolderthatwereinstalledaspartoftheservice(e.g.fileuploadsorcustomlogfiles).
Additionally,thedevelopmentmodewillleadtoaninconsistentstateoftheclusteruntilitisdisabled.WhileaserviceisrunningindevelopmentmodeyoucanmakechangestotheserviceonthefilesystemofanycoordinatorandseethemreflectedinrealtimejustlikewhenrunningArangoDBasasingleserver.Howeverthechangesmadeononecoordinatorwillnotbereflectedacrossothercoordinatorsuntilthedevelopmentmodeisdisabled.Whendisablingthedevelopmentmodeforaservice,thecoordinatorwillcreateanewbundleanddistributeitacrosstheservicelikeamanualupgradeoftheservice.
Forthesereasonsitisstronglyrecommendednottousedevelopmentmodeinaclusterwithmultiplecoordinatorsunlessyouaresurethatnorequestsorchangeswillbemadetoothercoordinatorswhileyouaremodifyingtheservice.Usingdevelopmentmodeinaproductionclusterisextremelyunsafeandhighlydiscouraged.
Ataglance
176
GettingStartedWe'regoingtostartwithanemptyfolder.Thiswillbetherootfolderofourservices.Youcannameitsomethingcleverbutforthecourseofthisguidewe'llassumeit'scalledthenameofyourservice:getting-started.
Firstweneedtocreateamanifest.Createanewfilecalledmanifest.jsonandaddthefollowingcontent:
{
"engines":{
"arangodb":"^3.0.0"
}
}
ThisjusttellsArangoDBtheserviceiscompatiblewithversions3.0.0andlater(allthewayuptobutnotincluding4.0.0),allowingolderversionsofArangoDBtounderstandthatthisservicelikelywon'tworkforthemandnewerversionswhatbehaviortoemulateshouldtheystillsupportit.
Thelittlehattotheleftoftheversionnumberisnotatypo,it'scalleda"caret"andindicatestheversionrange.Foxxusessemanticversioning(alsocalled"semver")formostofitsversionhandling.Youcanfindoutmoreabouthowsemverworksattheofficialsemverwebsite.
Nextwe'llneedtospecifyanentrypointtoourservice.ThisistheJavaScriptfilethatwillbeexecutedtodefineourservice'sHTTPendpoints.Wecandothisbyaddinga"main"fieldtoourmanifest:
{
"engines":{
"arangodb":"^3.0.0"
},
"main":"index.js"
}
That'sallweneedinourmanifestfornow,solet'snextcreatetheindex.jsfile:
'usestrict';
constcreateRouter=require('@arangodb/foxx/router');
constrouter=createRouter();
module.context.use(router);
Thefirstlinecausesourfiletobeinterpretedusingstrictmode.AllexamplesintheArangoDBdocumentationassumestrictmode,soyoumightwanttofamiliarizeyourselfwithitifyouhaven'tencountereditbefore.
Thesecondlineimportsthe@arangodb/foxx/routermodulewhichprovidesafunctionforcreatingnewFoxxrouters.We'reusingthisfunctiontocreateanewrouterobjectwhichwe'llbeusingforourservice.
Themodule.contextistheso-calledFoxxcontextorservicecontext.ThisvariableisavailableinallfilesthatarepartofyourFoxxserviceandprovidesaccesstoFoxxAPIsspecifictothecurrentservice,liketheusemethod,whichtellsFoxxtomounttherouterinthisservice(andtoexposeitsroutestoHTTP).
Nextlet'sdefinearoutethatprintsagenericgreeting:
//continued
router.get('/hello-world',function(req,res){
res.send('HelloWorld!');
})
.response(['text/plain'],'Agenericgreeting.')
.summary('Genericgreeting')
.description('Printsagenericgreeting.');
Therouterprovidesthemethodsget,post,etccorrespondingtoeachHTTPverbaswellasthecatch-allall.ThesemethodsindicatethatthegivenrouteshouldbeusedtohandleincomingrequestswiththegivenHTTPverb(oranymethodwhenusingall).
Gettingstarted
177
Thesemethodstakeanoptionalpath(ifomitted,itdefaultsto"/")aswellasarequesthandler,whichisafunctiontakingthereq(request)andres(response)objectstohandletheincomingrequestandgeneratetheoutgoingresponse.IfyouhaveusedtheexpressframeworkinNode.js,youmayalreadybefamiliarwithhowthisworks,otherwisecheckoutthechapteronroutes.
Theobjectreturnedbytherouter'smethodsprovidesadditionalmethodstoattachmetadataandvalidationtotheroute.We'reusingsummaryanddescriptiontodocumentwhattheroutedoes--thesearen'tstrictlynecessarybutgiveussomeniceauto-generateddocumentation.Theresponsemethodletsusadditionallydocumenttheresponsecontenttypeandwhattheresponsebodywillrepresent.
Tryitout
AtthispointyoucanuploadtheservicefolderasaziparchivefromthewebinterfaceusingtheServicestab.
ClickAddServicethenpicktheZipoptioninthedialog.Youwillneedtoprovideamountpath,whichistheURLprefixatwhichtheservicewillbemounted(e.g./getting-started).
Onceyouhavepickedtheziparchiveusingthefilepicker,theuploadshouldbeginimmediatelyandyourserviceshouldbeinstalled.OtherwisepresstheInstallbuttonandwaitforthedialogtodisappearandtheservicetoshowupintheservicelist.
Clickanywhereonthecardwithyourmountpathonthelabeltoopentheservice'sdetails.
IntheAPIdocumentationyoushouldseetheroutewedefinedearlier(/hello-world)withthewordGETnexttoitindicatingtheHTTPmethoditsupportsandthesummaryweprovidedontheright.Byclickingontheroute'spathyoucanopenthedocumentationfortheroute.
Notethatthedescriptionweprovidedappearsinthegenerateddocumentationaswellasthedescriptionweaddedtotheresponse(whichshouldcorrectlyindicatethecontenttypetext/plain,i.e.plaintext).
ClicktheTryitout!buttontosendarequesttotherouteandyoushouldseeanexamplerequestwiththeservice'sresponse:"HelloWorld!".
Congratulations!Youhavejustcreated,installedandusedyourfirstFoxxservice.
Parametervalidation
Let'saddanotherroutethatprovidesamorepersonalizedgreeting:
//continued
constjoi=require('joi');
router.get('/hello/:name',function(req,res){
res.send(`Hello${req.pathParams.name}`);
})
.pathParam('name',joi.string().required(),'Nametogreet.')
.response(['text/plain'],'Apersonalizedgreeting.')
.summary('Personalizedgreeting')
.description('Printsapersonalizedgreeting.');
ThefirstlineimportsthejoimodulefromnpmwhichcomesbundledwithArangoDB.JoiisavalidationlibrarythatisusedthroughoutFoxxtodefineschemasandparametertypes.
Note:Youcanbundleyourownmodulesfromnpmbyinstallingtheminyourservicefolderandmakingsurethenode_modulesfolderisincludedinyourziparchive.Formoreinformationseethesectiononmoduledependenciesinthechapterondependencies.
ThepathParammethodallowsustospecifyparametersweareexpectinginthepath.Thefirstargumentcorrespondstotheparameternameinthepath,thesecondargumentisajoischematheparameterisexpectedtomatchandthefinalargumentservestodescribetheparameterintheAPIdocumentation.
ThepathparametersareaccessiblefromthepathParamspropertyoftherequestobject.We'reusingatemplatestringtogeneratetheserver'sresponsecontainingtheparameter'svalue.
Gettingstarted
178
NotethatrouteswithpathparametersthatfailtovalidatefortherequestURLwillbeskippedasiftheywouldn'texist.Thisallowsyoutodefinemultipleroutesthatareonlydistinguishedbytheschemasoftheirpathparameters(e.g.aroutetakingonlynumericparametersandonetakinganystringasafallback).
Let'stakethisfurtherandcreatearoutethattakesaJSONrequestbody:
//continued
router.post('/sum',function(req,res){
constvalues=req.body.values;
res.send({
result:values.reduce(function(a,b){
returna+b;
},0)
});
})
.body(joi.object({
values:joi.array().items(joi.number().required()).required()
}).required(),'Valuestoaddtogether.')
.response(joi.object({
result:joi.number().required()
}).required(),'Sumoftheinputvalues.')
.summary('Addupnumbers')
.description('Calculatesthesumofanarrayofnumbervalues.');
Notethatweusedposttodefinethisrouteinsteadofget(whichdoesnotsupportrequestbodies).TryingtosendaGETrequesttothisroute'sURL(intheabsenceofagetrouteforthesamepath)willresultinFoxxrespondingwithanappropriateerrorresponse,indicatingthesupportedHTTPmethods.
AsthisroutenotonlyexpectsaJSONobjectasinputbutalsorespondswithaJSONobjectasoutputweneedtodefinetwoschemas.Wedon'tstrictlyneedaresponseschemabutithelpsdocumentingwhattherouteshouldbeexpectedtorespondwithandwillshowupintheAPIdocumentation.
Becausewe'repassingaschematotheresponsemethodwedon'tneedtoexplicitlytellFoxxwearesendingaJSONresponse.ThepresenceofaschemaintheabsenceofacontenttypealwaysimplieswewantJSON.Thoughwecouldjustadd["application/json"]asanadditionalargumentaftertheschemaifwewantedtomakethismoreexplicit.
Thebodymethodworksthesamewayastheresponsemethodexcepttheschemawillbeusedtovalidatetherequestbody.Iftherequestbodycan'tbeparsedasJSONordoesn'tmatchtheschema,Foxxwillrejecttherequestwithanappropriateerrorresponse.
Creatingcollections
TherealpowerofFoxxcomesfrominteractingwiththedatabaseitself.Inordertobeabletouseacollectionfromwithinourservice,weshouldfirstmakesurethatthecollectionactuallyexists.Therightplacetocreatecollectionsyourserviceisgoingtouseisinasetupscript,whichFoxxwillexecuteforyouwheninstallingorupdatingtheservice.
Firstcreateanewfoldercalled"scripts"intheservicefolder,whichwillbewhereourscriptsaregoingtolive.Forsimplicity'ssake,oursetupscriptwillliveinafilecalledsetup.jsinsidethatfolder:
//continued
'usestrict';
constdb=require('@arangodb').db;
constcollectionName='myFoxxCollection';
if(!db._collection(collectionName)){
db._createDocumentCollection(collectionName);
}
Thescriptusesthedbobjectfromthe@arangodbmodule,whichletsusinteractwiththedatabasetheFoxxservicewasinstalledinandthecollectionsinsidethatdatabase.Becausethescriptmaybeexecutedmultipletimes(i.e.wheneverweupdatetheserviceorwhentheserverisrestarted)weneedtomakesurewedon'taccidentallytrytocreatethesamecollectiontwice(whichwouldresultinanexception);wedothatbyfirstcheckingwhetheritalreadyexistsbeforecreatingit.
Gettingstarted
179
The_collectionmethodlooksupacollectionbynameandreturnsnullifnocollectionwiththatnamewasfound.The_createDocumentCollectionmethodcreatesanewdocumentcollectionbyname(_createEdgeCollectionalsoexistsandworksanalogouslyforedgecollections).
Note:Becausewehavehardcodedthecollectionname,multiplecopiesoftheserviceinstalledalongsideeachotherinthesamedatabasewillsharethesamecollection.Becausethismaynotalwaysbewhatyouwant,theFoxxcontextalsoprovidesthecollectionNamemethodwhichappliesamountpointspecificprefixtoanygivencollectionnametomakeituniquetotheservice.Italsoprovidesthecollectionmethod,whichbehavesalmostexactlylikedb._collectionexceptitalsoappliestheprefixbeforelookingthecollectionup.
Nextweneedtotellourserviceaboutthescriptbyaddingittothemanifestfile:
{
"engines":{
"arangodb":"^3.0.0"
},
"main":"index.js",
"scripts":{
"setup":"scripts/setup.js"
}
}
Theonlythingthathaschangedisthatweaddeda"scripts"fieldspecifyingthepathofthesetupscriptwejustwrote.
Gobacktothewebinterfaceandupdatetheservicewithournewcode,thenchecktheCollectionstab.Ifeverythingworkedright,youshouldseeanewcollectioncalled"myFoxxCollection".
AccessingcollectionsLet'sexpandourservicebyaddingafewmoreroutestoourindex.js:
//continued
constdb=require('@arangodb').db;
consterrors=require('@arangodb').errors;
constfoxxColl=db._collection('myFoxxCollection');
constDOC_NOT_FOUND=errors.ERROR_ARANGO_DOCUMENT_NOT_FOUND.code;
router.post('/entries',function(req,res){
constdata=req.body;
constmeta=foxxColl.save(req.body);
res.send(Object.assign(data,meta));
})
.body(joi.object().required(),'Entrytostoreinthecollection.')
.response(joi.object().required(),'Entrystoredinthecollection.')
.summary('Storeanentry')
.description('Storesanentryinthe"myFoxxCollection"collection.');
router.get('/entries/:key',function(req,res){
try{
constdata=foxxColl.document(req.pathParams.key);
res.send(data)
}catch(e){
if(!e.isArangoError||e.errorNum!==DOC_NOT_FOUND){
throwe;
}
res.throw(404,'Theentrydoesnotexist',e);
}
})
.pathParam('key',joi.string().required(),'Keyoftheentry.')
.response(joi.object().required(),'Entrystoredinthecollection.')
.summary('Retrieveanentry')
.description('Retrievesanentryfromthe"myFoxxCollection"collectionbykey.');
We'reusingthesaveanddocumentmethodsofthecollectionobjecttostoreandretrievedocumentsinthecollectionwecreatedinoursetupscript.Becausewedon'tcarewhatthedocumentslooklikeweallowanyattributesontherequestbodyandjustacceptanobject.
Gettingstarted
180
BecausethekeywillbeautomaticallygeneratedbyArangoDBwhenonewasn'tspecifiedintherequestbody,we'reusingObject.assigntoapplytheattributesofthemetadataobjectreturnedbythesavemethodtothedocumentbeforereturningitfromourfirstroute.
Thedocumentmethodreturnsadocumentinacollectionbyits_keyor_id.HoweverwhennomatchingdocumentexistsitthrowsanArangoErrorexception.BecausewewanttoprovideamoredescriptiveerrormessagethanArangoDBdoesoutofthebox,weneedtohandlethaterrorexplicitly.
AllArangoErrorexceptionshaveatruthyattributeisArangoErrorthathelpsyourecognizingtheseerrorswithouthavingtoworryaboutinstanceofchecks.TheyalsoprovideanerrorNumandanerrorMessage.Ifyouwanttocheckforspecificerrorsyoucanjustimporttheerrorsobjectfromthe@arangodbmoduleinsteadofhavingtomemorizenumericerrorcodes.
Insteadofdefiningourownresponselogicfortheerrorcasewejustuseres.throw,whichmakestheresponseobjectthrowanexceptionFoxxcanrecognizeandconverttotheappropriateserverresponse.WealsopassalongtheexceptionitselfsoFoxxcanprovidemorediagnosticinformationwhenwewantitto.
Wecouldextendthepostroutetosupportarraysofobjectsaswell,eachobjectfollowingacertainschema:
//storeschemainvariabletomakeitre-usable,see.body()
constdocSchema=joi.object().required().keys({
name:joi.string().required(),
age:joi.number().required()
}).unknown();//allowadditionalattributes
router.post('/entries',function(req,res){
constmultiple=Array.isArray(req.body);
constbody=multiple?req.body:[req.body];
letdata=[];
for(vardocofbody){
constmeta=foxxColl.save(doc);
data.push(Object.assign(doc,meta));
}
res.send(multiple?data:data[0]);
})
.body(joi.alternatives().try(
docSchema,
joi.array().items(docSchema)
),'Entryorentriestostoreinthecollection.')
.response(joi.alternatives().try(
joi.object().required(),
joi.array().items(joi.object().required())
),'Entryorentriesstoredinthecollection.')
.summary('Storeentryorentries')
.description('Storeasingleentryormultipleentriesinthe"myFoxxCollection"collection.');
WritingdatabasequeriesStoringandretrievingentriesisfine,butrightnowwehavetomemorizeeachkeywhenwecreateanentry.Let'saddaroutethatgivesusalistofthekeysofallentriessowecanusethosetolookanentryupindetail.
ThenaïveapproachwouldbetousethetoArray()methodtoconverttheentirecollectiontoanarrayandjustreturnthat.Butwe'reonlyinterestedinthekeysandtheremightpotentiallybesomanyentriesthatfirstretrievingeverysingledocumentmightgetunwieldy.Let'swriteashortAQLquerytodothisinstead:
//continued
constaql=require('@arangodb').aql;
router.get('/entries',function(req,res){
constkeys=db._query(aql`
FORentryIN${foxxColl}
RETURNentry._key
`);
res.send(keys);
})
.response(joi.array().items(
joi.string().required()
Gettingstarted
181
).required(),'Listofentrykeys.')
.summary('Listentrykeys')
.description('Assemblesalistofkeysofentriesinthecollection.');
Herewe'reusingtwonewthings:
The_querymethodexecutesanAQLqueryintheactivedatabase.
Theaqltemplatestringhandlerallowsustowritemulti-lineAQLqueriesandalsohandlesqueryparametersandcollectionnames.InsteadofhardcodingthenameofthecollectionwewanttouseinthequerywecansimplyreferencethefoxxCollvariablewedefinedearlier--itrecognizesthevalueasanArangoDBcollectionobjectandknowswearespecifyingacollectionratherthanaregularvalueeventhoughAQLdistinguishesbetweenthetwo.
Note:Ifyouaren'tusedtoJavaScripttemplatestringsandtemplatestringhandlersjustthinkofaqlasafunctionthatreceivesthemultilinestringsplitatevery${}expressionaswellasanarrayofthevaluesofthoseexpressions--that'sactuallyallthereistoit.
Alternatively,here'saversionwithouttemplatestrings(noticehowmuchcleanertheaqlversionwillbeincomparisonwhenyouhavemultiplevariables):
constkeys=db._query(
'FORentryIN@@collRETURNentry._key',
{'@coll':foxxColl}
);
Nextsteps
YounowknowhowtocreateaFoxxservicefromscratch,howtohandleuserinputandhowtoaccessthedatabasefromwithinyourFoxxservicetostore,retrieveandquerydatayoustoreinsideArangoDB.ThisshouldallowyoutobuildmeaningfulAPIsforyourownapplicationsbuttherearemanymorethingsyoucandowithFoxx:
Needtogofaster?Turnondevelopmentmodeandhackonyourcoderightontheserver.
Concernedaboutsecurity?Youcouldaddauthenticationtoyourservicetoprotectaccesstothedatabeforeitevenleavesthedatabase.
Writingasinglepageapp?YoucouldstoresomebasicassetsrightinsideyourFoxxservice.
Needtointegrateexternalservices?YoucanmakeHTTPrequestsfrominsideFoxxandusequeuedjobstoperformthatworkinthebackground.
Tiredofreinventingthewheel?Learnaboutdependencies.
Everythingbroken?Youcanwriteteststomakesureyourlogicremainssound.
Gettingstarted
182
ManifestfilesEveryservicecomeswithamanifest.jsonfileprovidingmetadata.Thefollowingfieldsareallowedinmanifests:
configuration:Object(optional)
Anobjectdefiningtheconfigurationoptionsthisservicerequires.
defaultDocument:string(optional)
Ifspecified,the/(root)routeoftheservicewillautomaticallyredirecttothegivenrelativepath,e.g.:
"defaultDocument":"index.html"
ThiswouldhavethesameeffectascreatingthefollowingrouteinJavaScript:
constcreateRouter=require('@arangodb/foxx/router');
constindexRouter=createRouter();
indexRouter.all('/',function(req,res){
res.redirect('index.html');
});
module.context.use(indexRouter);
Note:Asof3.0.0thisfieldcansafelybeomitted;thevaluenolongerdefaultsto"index.html".
dependencies:Object(optional)andprovides:Object(optional)
Objectsspecifyingotherservicesthisservicehasasdependenciesandwhatdependenciesitcanprovidetootherservices.
engines:Object(optional)
AnobjectindicatingthesemanticversionrangesofArangoDB(orcompatibleenvironments)theservicewillbecompatiblewith,e.g.:
"engines":{
"arangodb":"^3.0.0"
}
ThisshouldcorrectlyindicatetheminimumversionofArangoDBtheservicehasbeentestedagainst.FoxxmaintainsastrictsemanticversioningpolicyasofArangoDB3.0.0soitisgenerallysafetousesemverranges(e.g. 3.0.0tomatchanyversiongreaterorequalto3.0.0andbelow4.0.0)formaximumcompatibility.
files:Object(optional)
Anobjectdefiningfileassetsservedbythisservice.
lib:string(Default:".")
TherelativepathtotheFoxxJavaScriptfilesintheservice,e.g.:
"lib":"lib"
Thiswouldresultinthemainentrypoint(seebelow)andotherJavaScriptpathsbeingresolvedasrelativetothelibfolderinsidetheservicefolder.
main:string(optional)
Therelativepathtothemainentrypointofthisservice(relativetolib,seeabove),e.g.:
"main":"index.js"
ThiswouldresultinFoxxloadingandexecutingthefileindex.jswhentheserviceismountedorstarted.
Servicemanifest
183
Note:whileitistechnicallypossibletoomitthisfield,youwilllikelywanttoprovideanentrypointtoyourserviceasthisistheonlywaytoexposeHTTProutesorexportaJavaScriptAPI.
scripts:Object(optional)
Anobjectdefiningnamedscriptsprovidedbythisservice,whichcaneitherbeuseddirectlyorasqueuedjobsbyotherservices.
tests:stringorArray<string>(optional)
ApathorlistofpathsofJavaScripttestsprovidedforthisservice.
Additionallymanifestscanprovidethefollowingmetadata:
author:string(optional)
Thefullnameoftheauthoroftheservice(i.e.you).Thiswillbeshowninthewebinterface.
contributors:Array<string>(optional)
Alistofnamesofpeoplethathavecontributedtothedevelopmentoftheserviceinsomeway.Thiswillbeshowninthewebinterface.
description:string(optional)
Ahuman-readabledescriptionoftheservice.Thiswillbeshowninthewebinterface.
keywords:Array<string>(optional)
Alistofkeywordsthathelpcategorizethisservice.ThisisusedbytheFoxxStoreinstallerstoorganizeservices.
license:string(optional)
Astringidentifyingthelicenseunderwhichtheserviceispublished,ideallyintheformofanSPDXlicenseidentifier.Thiswillbeshowninthewebinterface.
name:string(optional)
ThenameoftheFoxxservice.AllowedcharactersareA-Z,0-9,theASCIIhyphen(-)andunderscore(_)characters.Thenamemustnotstartwithanumber.Thiswillbeshowninthewebinterface.
thumbnail:string(optional)
Thefilenameofathumbnailthatwillbeusedalongsidetheserviceinthewebinterface.ThisshouldbeaJPEGorPNGimagethatlooksgoodatsizes50x50and160x160.
version:string(optional)
TheversionnumberoftheFoxxservice.Theversionnumbermustfollowthesemanticversioningformat.Thiswillbeshowninthewebinterface.
Examples
{
"name":"example-foxx-service",
"version":"3.0.0-dev",
"license":"MIT",
"description":"Anexampleservicewitharelativelyfull-featuredmanifest.",
"thumbnail":"foxx-icon.png",
"keywords":["demo","service"],
"author":"ArangoDBGmbH",
"contributors":[
"AlanPlum<[email protected]>"
],
"lib":"dist",
"main":"entry.js",
"defaultDocument":"welcome.html",
"engines":{
"arangodb":"^3.0.0"
},
"files":{
Servicemanifest
184
"welcome.html":"assets/index.html",
"hello.jpg":"assets/hello.jpg"
"world.jpg":{
"path":"assets/world.jpg",
"type":"image/jpeg",
"gzip":false
}
},
"tests":"dist/**.spec.js"
}
Servicemanifest
185
FoxxservicecontextTheservicecontextprovidesaccesstomethodsandattributesthatarespecifictoagivenservice.InaFoxxservicethecontextisgenerallyavailableasthemodule.contextvariable.Withinarouter'srequesthandlertherequestandresponseobjects'contextattributealsoprovideaccesstothecontextoftheservicetheroutewasmountedin(whichmaybedifferentfromtheonetheroutehandlerwasdefinedin).
Examples
//inservice/my-foxx-1
constcreateRouter=require('@arangodb/foxx/router');
constrouter=createRouter();
//Seethechapterondependenciesformoreinfoon
//howexportsanddependenciesworkacrossservices
module.exports={routes:router};
router.get(function(req,res){
module.context.mount==='/my-foxx-1';
req.context.mount==='/my-foxx-2';
res.write('Hellofrommy-foxx-1');
});
//inservice/my-foxx-2
constcreateRouter=require('@arangodb/foxx/router');
constrouter2=createRouter();
module.context.use(router2);
router2.post(function(req,res){
module.context.mount==='/my-foxx-2';
req.context.mount==='/my-foxx-2';
res.write('Hellofrommy-foxx-2');
});
constrouter1=module.context.dependencies.myFoxx1.routes;
module.context.use(router1);
Theservicecontextspecifiesthefollowingproperties:
argv:any
Anyargumentspassedinifthecurrentfilewasexecutedasascriptorqueuedjob.
basePath:string
Thefilesystempathoftheservice,i.e.thefolderinwhichtheservicewasinstalledtobyArangoDB.
baseUrl:string
ThebaseURLoftheservice,relativetotheArangoDBserver,e.g./_db/_system/my-foxx.
collectionPrefix:string
TheprefixthatwillbeusedbycollectionandcollectionNametoderivethenamesofservice-specificcollections.Thisisderivedfromtheservice'smountpoint,e.g./my-foxxbecomesmy_foxx.
configuration:Object
Configurationoptionsfortheservice.
dependencies:Object
Configureddependenciesfortheservice.
isDevelopment:boolean
Indicateswhethertheserviceisrunningindevelopmentmode.
Servicecontext
186
isProduction:boolean
TheinverseofisDevelopment.
manifest:Object
Theparsedmanifestfileoftheservice.
mount:string
Themountpointoftheservice,e.g./my-foxx.
apiDocumentationmodule.context.apiDocumentation([options]):Function
DEPRECATED
CreatesarequesthandlerthatservestheAPIdocumentation.
Note:ThismethodhasbeendeprecatedinArangoDB3.1andreplacedwiththemorestraightforwardcreateDocumentationRoutermethodprovidingthesamefunctionality.
Arguments
SeecreateDocumentationRouterbelow.
Examples
//ServetheAPIdocsforthecurrentservice
router.get('/docs/*',module.context.apiDocumentation());
//Notethatthepathmustendwithawildcard
//andtheroutemustuseHTTPGET.
createDocumentationRoutermodule.context.createDocumentationRouter([options]):Router
CreatesarouterthatservestheAPIdocumentation.
Note:Theroutercanbemountedlikeanyotherchildrouter(seeexamplesbelow).
Arguments
options:Object(optional)
Anobjectwithanyofthefollowingproperties:
mount:string(Default:module.context.mount)
Themountpathoftheservicetoservethedocumentationof.
indexFile:string(Default:"index.html")
FilenameoftheHTMLfileservingtheAPIdocumentation.
swaggerRoot:string(optional)
FullpathofthefoldercontainingtheSwaggerassetsandtheindexFile.DefaultstotheSwaggerassetsusedbythewebinterface.
before:Function(optional)
Afunctionthatwillbeexecutedbeforearequestishandled.
Ifthefunctionreturnsfalsetherequestwillnotbeprocessedanyfurther.
Ifthefunctionreturnsanobject,itsattributeswillbeusedtooverridetheoptionsforthecurrentrequest.
Servicecontext
187
Anyotherreturnvaluewillbeignored.
Ifoptionsisafunctionitwillbeusedasthebeforeoption.
IfoptionsisastringitwillbeusedastheswaggerRootoption.
ReturnsaFoxxrouter.
Examples
//ServetheAPIdocsforthecurrentservice
router.use('/docs',module.context.createDocumentationRouter());
//--or--
//ServetheAPIdocsfortheservicetherouterismountedin
router.use('/docs',module.context.createDocumentationRouter(function(req){
return{mount:req.context.mount};
}));
//--or--
//ServetheAPIdocsonlyforusersauthenticatedwithArangoDB
router.use('/docs',module.context.createDocumentationRouter(function(req,res){
if(req.suffix==='swagger.json'&&!req.arangoUser){
res.throw(401,'Notauthenticated');
}
}));
collectionmodule.context.collection(name):ArangoCollection|null
PassesthegivennametocollectionName,thenlooksupthecollectionwiththeprefixedname.
Arguments
name:string
Unprefixednameoftheservice-specificcollection.
Returnsacollectionornullifnocollectionwiththeprefixednameexists.
collectionNamemodule.context.collectionName(name):string
PrefixesthegivennamewiththecollectionPrefixforthisservice.
Arguments
name:string
Unprefixednameoftheservice-specificcollection.
Returnstheprefixedname.
Examples
module.context.mount==='/my-foxx'
module.context.collectionName('doodads')==='my_foxx_doodads'
filemodule.context.file(name,[encoding]):Buffer|string
PassesthegivennametofileName,thenloadsthefilewiththeresultingname.
Servicecontext
188
Arguments
name:string
Nameofthefiletoload,relativetothecurrentservice.
encoding:string(optional)
Encodingofthefile,e.g.utf-8.Ifomittedthefilewillbeloadedasarawbufferinsteadofastring.
Returnsthefile'scontents.
fileNamemodule.context.fileName(name):string
Resolvesthegivenfilenamerelativetothecurrentservice.
Arguments
name:string
Nameofthefile,relativetothecurrentservice.
Returnstheabsolutefilepath.
usemodule.context.use([path],router):Endpoint
Mountsagivenrouterontheservicetoexposetherouter'sroutesontheservice'smountpoint.
Arguments
path:string(Default:"/")
Pathtomounttherouterat,relativetotheservice'smountpoint.
router:Router|Middleware
Arouterormiddlewaretomount.
ReturnsanEndpointforthegivenrouterormiddleware.
Note:Mountingservicesatruntime(e.g.withinrequesthandlersorqueuedjobs)isnotsupported.
Servicecontext
189
FoxxconfigurationFoxxservicescandefineconfigurationparameterstomakethemmorere-usable.
Theconfigurationobjectmapsnamestoconfigurationparameters:
Thekeyisthenameunderwhichtheparameterwillbeavailableontheservicecontext'sconfigurationproperty.
Thevalueisaparameterdefinition.
Theparameterdefinitioncanhavethefollowingproperties:
description:string
Humanreadabledescriptionoftheparameter.
type:string(Default:"string")
Typeoftheconfigurationparameter.Supportedvaluesare:
"integer"or"int":anyfiniteintegernumber.
"boolean"or"bool":thevaluestrueorfalse.
"number":anyfinitedecimalorintegernumber.
"string":anystringvalue.
"json":anywell-formedJSONvalue.
"password":likestringbutwillbedisplayedasamaskedinputfieldinthewebfrontend.
default:any
Defaultvalueoftheconfigurationparameter.
required:(Default:true)
Whethertheparameterisrequired.
Iftheconfigurationhasparametersthatdonotspecifyadefaultvalue,youneedtoconfiguretheservicebeforeitbecomesactive.InthemeantimeafallbackservicelicationwillbemountedthatrespondstoallrequestswithaHTTP500statuscodeindicatingaserver-sideerror.
TheconfigurationparametersofamountedservicecanbeadjustedfromthewebinterfacebyclickingtheConfigurationbuttonintheservicedetails.
Examples
"configuration":{
"currency":{
"description":"Currencysymboltouseforpricesintheshop.",
"default":"$",
"type":"string"
},
"secretKey":{
"description":"Secretkeytouseforsigningsessiontokens.",
"type":"password"
}
}
Configuration
190
DependencymanagementTherearetwothingscommonlycalled"dependencies"inFoxx:
Moduledependencies,i.e.dependenciesonexternalJavaScriptmodules(e.g.fromthepublicnpmregistry)
Foxxdependencies,i.e.dependenciesbetweenFoxxservices
Let'slookattheminmoredetail:
Moduledependencies
Youcanusethenode_modulesfoldertobundlethird-partyFoxx-compatiblenpmandNode.jsmoduleswithyourFoxxservice.Typicallythisisachievedbyaddingapackage.jsonfiletoyourprojectspecifyingnpmdependenciesusingthedependenciesattributeandinstallingthemwiththenpmcommand-linetool.
Makesuretoincludetheactualnode_modulesfolderinyourFoxxservicebundleasArangoDBwillnotdoanythingspecialtoinstallthesedependencies.AlsokeepinmindthatbundlingextraneousmoduleslikedevelopmentdependenciesmaybloatthefilesizeofyourFoxxservicebundle.
Compatibilitycaveats
UnlikeJavaScriptinbrowsersorNode.js,theJavaScriptenvironmentinArangoDBissynchronous.ThismeansanymodulesthatdependonasynchronousbehaviourlikepromisesorsetTimeoutwillnotbehavecorrectlyinArangoDBorFoxx.AdditionallyunlikeNode.jsArangoDBdoesnotsupportnativeextensions.AllmoduleshavetobeimplementedinpureJavaScript.
WhileArangoDBprovidesalotofcompatibilitycodetosupportmoduleswrittenforNode.js,someNode.jsbuilt-inmodulescannotbeprovidedbyArangoDB.ForacloserlookattheNode.jsmodulesArangoDBdoesordoesnotprovidecheckouttheappendixonJavaScriptmodules.
Alsonotethattheserestrictionsnotonlyapplyonthemodulesyouwishtoinstallbutalsothedependenciesofthosemodules.Asaruleofthumb:moduleswrittentoworkinNode.jsandthebrowserthatdonotrelyonasyncbehaviourshouldgenerallywork;modulesthatrelyonnetworkorfilesystemI/Oormakeheavyuseofasyncbehaviourmostlikelywillnot.
Foxxdependencies
Foxxdependenciescanbedeclaredinaservice'smanifestusingtheprovidesanddependenciesfields:
providesliststhedependenciesagivenserviceprovides,i.e.whichAPIsitclaimstobecompatiblewith
dependenciesliststhedependenciesagivenserviceuses,i.e.whichAPIsitsdependenciesneedtobecompatiblewith
Adependencynameshouldgenerallyusethesameformatasanamespaced(org-scoped)NPMmodule,e.g.@foxx/sessions.
DependencynamesrefertotheexternalJavaScriptAPIofaserviceratherthanspecificservicesimplementingthoseAPIs.Somedependencynamesdefinedbyofficiallymaintainedservicesare:
@foxx/auth(version1.0.0)@foxx/api-keys(version1.0.0)@foxx/bugsnag(versions1.0.0and2.0.0)@foxx/mailgun(versions1.0.0and2.0.0)@foxx/postageapp(versions1.0.0and2.0.0)@foxx/postmark(versions1.0.0and2.0.0)@foxx/sendgrid(versions1.0.0and2.0.0)@foxx/oauth2(versions1.0.0and2.0.0)@foxx/segment-io(versions1.0.0and2.0.0)@foxx/sessions(versions1.0.0and2.0.0)@foxx/users(versions1.0.0,2.0.0and3.0.0)
Dependencies
191
Aprovidesdefinitionmapseachprovideddependency'snametotheprovidedversion:
"provides":{
"@foxx/auth":"1.0.0"
}
Adependenciesdefinitionmapsthelocalaliasofagivendependencyagainstitsnameandthesupportedversionrange(eitherasaJSONobjectorashorthandstring):
"dependencies":{
"mySessions":"@foxx/sessions:^2.0.0",
"myAuth":{
"name":"@foxx/auth",
"version":"^1.0.0",
"description":"Thisdescriptionisentirelyoptional.",
"required":false
}
}
Dependenciescanbeconfiguredfromthewebinterfaceinaservice'ssettingstabusingtheDependenciesbutton.
Thevalueforeachdependencyshouldbethedatabase-relativemountpathoftheservice(includingtheleadingslash).Inordertobeusableasthedependencyofanotherservicebothservicesneedtobemountedinthesamedatabase.Aservicecanbeusedtoprovidemultipledependenciesforthesameservice(aslongastheexpectedJavaScriptAPIsdon'tconflict).
Aservicethathasunconfiguredrequireddependenciescannotbeuseduntilallofitsdependencieshavebeenconfigured.
Itispossibletospecifythemountpathofaservicethatdoesnotactuallydeclarethedependencyasprovided.Thereiscurrentlynovalidationbeyondthemanifestformats.
Whenaserviceusesanothermountedserviceasadependencythedependency'smainentryfile'sexportsobjectbecomesavailableinthemodule.context.dependenciesobjectoftheotherservice:
Examples
ServiceAandServiceBaremountedinthesamedatabase.ServiceBhasadependencywiththelocalalias"greeter".ThedependencyisconfiguredtousethemountpathofServiceA.
//EntryfileofServiceA
module.exports={
sayHi(){
return'Hello';
}
};
//SomewhereinServiceB
constgreeter=module.context.dependencies.greeter;
res.write(greeter.sayHi());
Dependencies
192
RoutersconstcreateRouter=require('@arangodb/foxx/router');
RoutersletyoudefineroutesthatextendArangoDB'sHTTPAPIwithcustomendpoints.
RoutersneedtobemountedusingtheusemethodofaservicecontexttoexposetheirHTTProutesataservice'smountpath.
Youcanpassroutersbetweenservicesmountedinthesamedatabaseasdependencies.Youcanevennestrouterswithineachother.
CreatingaroutercreateRouter():Router
Thisreturnsanew,cleanrouterobjectthathasnotyetbeenmountedintheserviceandcanbeexportedlikeanyotherobject.
Requesthandlersrouter.get([path],[...middleware],handler,[name]):Endpoint
router.post([path],[...middleware],handler,[name]):Endpoint
router.put([path],[...middleware],handler,[name]):Endpoint
router.patch([path],[...middleware],handler,[name]):Endpoint
router.delete([path],[...middleware],handler,[name]):Endpoint
router.all([path],[...middleware],handler,[name]):Endpoint
Thesemethodsletyouspecifyroutesontherouter.TheallmethoddefinesaroutethatwillmatchanysupportedHTTPverb,theothermethodsdefineroutesthatonlymatchtheHTTPverbwiththesamename.
Arguments
path:string(Default:"/")
ThepathoftherequesthandlerrelativetothebasepaththeRouterismountedat.Ifomitted,therequesthandlerwillhandlerequeststothebasepathoftheRouter.Forinformationondefiningdynamicroutesseethesectiononpathparametersinthechapteronrouterendpoints.
middleware:Function(optional)
Zeroormoremiddlewarefunctionsthattakethefollowingarguments:
req:Request
Anincomingserverrequestobject.
res:Response
Anoutgoingserverresponseobject.
next:Function
Acallbackthatpassescontrolovertothenextmiddlewarefunctionandreturnswhenthatfunctionhascompleted.
Ifatruthyargumentispassed,thatargumentwillbethrownasanerror.
Ifthereisnonextmiddlewarefunction,thehandlerwillbeinvokedinstead(seebelow).
handler:Function
Afunctionthattakesthefollowingarguments:
req:Request
Anincomingserverrequestobject.
Routers
193
res:Response
Anoutgoingserverresponse.
name:string(optional)
AnamethatcanbeusedtogenerateURLsfortheendpoint.Formoreinformationseethereversemethodoftherequestobject.
ReturnsanEndpointfortheroute.
Examples
Simpleindexroute:
router.get(function(req,res){
res.set('content-type','text/plain');
res.write('HelloWorld!');
});
RestrictingaccesstoauthenticatedArangoDBusers:
router.get('/secrets',function(req,res,next){
if(req.arangoUser){
next();
}else{
res.throw(404,'Secrets?Whatsecrets?');
}
},function(req,res){
res.download('allOurSecrets.zip');
});
Multiplemiddlewarefunctions:
functioncounting(req,res,next){
if(!req.counter)req.counter=0;
req.counter++;
next();
req.counter--;
}
router.get(counting,counting,counting,function(req,res){
res.json({counter:req.counter});//{"counter":3}
});
Mountingchildroutersandmiddlewarerouter.use([path],middleware,[name]):Endpoint
Theusemethodletsyoumountachildrouterormiddlewareatagivenpath.
Arguments
path:string(optional)
ThepathofthemiddlewarerelativetothebasepaththeRouterismountedat.Ifomitted,themiddlewarewillhandlerequeststothebasepathoftheRouter.Forinformationondefiningdynamicroutesseethesectiononpathparametersinthechapteronrouterendpoints.
middleware:Router|Middleware
Anunmountedrouterobjectoramiddleware.
name:string(optional)
AnamethatcanbeusedtogenerateURLsforendpointsofthisrouter.Formoreinformationseethereversemethodoftherequestobject.HasnoeffectifhandlerisaMiddleware.
ReturnsanEndpointforthemiddlewareorchildrouter.
Routers
194
Routers
195
EndpointsEndpointsarereturnedbytheuse,allandHTTPverb(e.g.get,post)methodsofroutersaswellastheusemethodoftheservicecontext.Theycanbeusedtoattachmetadatatomountedroutes,middlewareandchildroutersthataffectshowrequestsandresponsesareprocessedorprovidesAPIdocumentation.
Endpointsshouldonlybeusedtoinvokethefollowingmethods.Endpointmethodscanbechainedtogether(eachmethodreturnstheendpointitself).
headerendpoint.header(name,[schema],[description]):this
Definesarequestheaderrecognizedbytheendpoint.Anyadditionalnon-definedheaderswillbetreatedasoptionalstringvalues.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisheaderdefinitionunlessoverridden.
Arguments
name:string
Nameoftheheader.Thisshouldbeconsideredcaseinsensitiveasallheadernameswillbeconvertedtolowercase.
schema:Schema(optional)
Aschemadescribingtheformatoftheheadervalue.Thiscanbeajoischemaoranythingthathasacompatiblevalidatemethod.
Thevalueofthisheaderwillbesettothevaluepropertyofthevalidationresult.Avalidationfailurewillresultinanautomatic400(BadRequest)errorresponse.
description:string(optional)
AhumanreadablestringthatwillbeshownintheAPIdocumentation.
Returnstheendpoint.
Examples
router.get(/*...*/)
.header('arangoVersion',joi.number().min(30000).default(30000));
pathParamendpoint.pathParam(name,[schema],[description]):this
Definesapathparameterrecognizedbytheendpoint.Pathparametersareexpectedtobefilledaspartoftheendpoint'smountpath.Anyadditionalnon-definedpathparameterswillbetreatedasoptionalstringvalues.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisparameterdefinitionunlessoverridden.
Arguments
name:string
Nameoftheparameter.
schema:Schema(optional)
Aschemadescribingtheformatoftheparameter.Thiscanbeajoischemaoranythingthathasacompatiblevalidatemethod.
Endpoints
196
Thevalueofthisparameterwillbesettothevaluepropertyofthevalidationresult.Avalidationfailurewillresultintheroutefailingtomatchandbeingignored(resultingina404(NotFound)errorresponseifnootherroutesmatch).
description:string(optional)
AhumanreadablestringthatwillbeshownintheAPIdocumentation.
Returnstheendpoint.
Examples
router.get('/some/:num/here',/*...*/)
.pathParam('num',joi.number().required());
queryParamendpoint.queryParam(name,[schema],[description]):this
Definesaqueryparameterrecognizedbytheendpoint.Anyadditionalnon-definedqueryparameterswillbetreatedasoptionalstringvalues.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisparameterdefinitionunlessoverridden.
Arguments
name:string
Nameoftheparameter.
schema:Schema(optional)
Aschemadescribingtheformatoftheparameter.Thiscanbeajoischemaoranythingthathasacompatiblevalidatemethod.
Thevalueofthisparameterwillbesettothevaluepropertyofthevalidationresult.Avalidationfailurewillresultinanautomatic400(BadRequest)errorresponse.
description:string(optional)
AhumanreadablestringthatwillbeshownintheAPIdocumentation.
Returnstheendpoint.
Examples
router.get(/*...*/)
.queryParam('num',joi.number().required());
bodyendpoint.body([model],[mimes],[description]):this
Definestherequestbodyrecognizedbytheendpoint.Therecanonlybeonerequestbodydefinitionperendpoint.ThedefinitionwillalsobeshownintheroutedetailsintheAPIdocumentation.
Intheabsenceofarequestbodydefinition,therequestobject'sbodypropertywillbeinitializedtotheunprocessedrawBodybuffer.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisbodydefinitionunlessoverridden.Iftheendpointisamiddleware,therequestbodywillonlybeparsedonce(i.e.theMIMEtypesoftheroutematchingthesamerequestwillbeignoredbutthebodywillstillbevalidatedagain).
Arguments
model:Model|Schema|null(optional)
Amodelorjoischemadescribingtherequestbody.Avalidationfailurewillresultinanautomatic400(BadRequest)errorresponse.
Endpoints
197
IfthevalueisamodelwithafromClientmethod,thatmethodwillbeappliedtotheparsedrequestbody.
Ifthevalueisaschemaoramodelwithaschema,theschemawillbeusedtovalidatetherequestbodyandthevaluepropertyofthevalidationresultoftheparsedrequestbodywillbeusedinsteadoftheparsedrequestbodyitself.
IfthevalueisamodeloraschemaandtheMIMEtypehasbeenomitted,theMIMEtypewilldefaulttoJSONinstead.
Ifthevalueisexplicitlysettonull,norequestbodywillbeexpected.
Ifthevalueisanarraycontainingexactlyonemodelorschema,therequestbodywillbetreatedasanarrayofitemsmatchingthatmodelorschema.
mimes:Array<string>(optional)
AnarrayofMIMEtypestheroutesupports.
Commonnon-mimealiaseslike"json"or"html"arealsosupportedandwillbeexpandedtotheappropriateMIMEtype(e.g."application/json"and"text/html").
IftheMIMEtypeisrecognizedbyFoxxtherequestbodywillbeparsedintotheappropriatestructurebeforebeingvalidated.CurrentlyonlyJSON,application/x-www-form-urlencodedandmultipartformatsaresupportedinthisway.
IftheMIMEtypeindicatedintherequestheadersdoesnotmatchanyofthesupportedMIMEtypes,thefirstMIMEtypeinthelistwillbeusedinstead.
Failuretoparsetherequestbodywillresultinanautomatic400(BadRequest)errorresponse.
description:string(optional)
AhumanreadablestringthatwillbeshownintheAPIdocumentation.
Returnstheendpoint.
Examples
router.post('/expects/some/json',/*...*/)
.body(
joi.object().required(),
'ThisimpliesJSON.'
);
router.post('/expects/nothing',/*...*/)
.body(null);//Nobodyallowed
router.post('/expects/some/plaintext',/*...*/)
.body(['text/plain'],'Thisbodywillbeastring.');
responseendpoint.response([status],[model],[mimes],[description]):this
Definesaresponsebodyfortheendpoint.Whenusingtheresponseobject'ssendmethodintherequesthandlerofthisroute,thedefinitionwiththematchingstatuscodewillbeusedtogeneratetheresponsebody.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisresponsedefinitionunlessoverridden.Iftheendpointisamiddleware,thismethodhasnoeffect.
Arguments
status:number|string(Default:200or204)
HTTPstatuscodetheresponseappliesto.Ifastringisprovidedinsteadofanumericstatuscodeitwillbeusedtolookupanumericstatuscodeusingthestatusesmodule.
model:Model|Schema|null(optional)
Amodelorjoischemadescribingtheresponsebody.
Endpoints
198
IfthevalueisamodelwithaforClientmethod,thatmethodwillbeappliedtothedatapassedtoresponse.sendwithintherouteiftheresponsestatuscodematches(butalsoifnostatuscodehasbeenset).
Ifthevalueisaschemaoramodelwithaschema,theactualschemawillnotbeusedtovalidatetheresponsebodyandonlyservestodocumenttheresponseinmoredetailintheAPIdocumentation.
IfthevalueisamodeloraschemaandtheMIMEtypehasbeenomitted,theMIMEtypewilldefaulttoJSONinstead.
Ifthevalueisexplicitlysettonullandthestatuscodehasbeenomitted,thestatuscodewilldefaultto204("nocontent")insteadof200.
Ifthevalueisanarraycontainingexactlyonemodelorschema,theresponsebodywillbeanarrayofitemsmatchingthatmodelorschema.
mimes:Array<string>(optional)
AnarrayofMIMEtypestheroutemightrespondwithforthisstatuscode.
Commonnon-mimealiaseslike"json"or"html"arealsosupportedandwillbeexpandedtotheappropriateMIMEtype(e.g."application/json"and"text/html").
Whenusingtheresponse.sendmethodtheresponsebodywillbeconvertedtotheappropriateMIMEtypeifpossible.
description:string(optional)
Ahuman-readablestringthatbrieflydescribestheresponseandwillbeshownintheendpoint'sdetaileddocumentation.
Returnstheendpoint.
Examples
//Thisexampleonlyprovidesdocumentation
//andimpliesagenericJSONresponsebody.
router.get(/*...*/)
.response(
joi.array().items(joi.string()),
'Alistofdoodadidentifiers.'
);
//Noresponsebodywillbeexpectedhere.
router.delete(/*...*/)
.response(null,'Thedoodadnolongerexists.');
//Anendpointcandefinemultipleresponsetypes
//fordifferentstatuscodes--butnevermorethan
//oneforeachstatuscode.
router.post(/*...*/)
.response('found','Thedoodadislocatedelsewhere.')
.response(201,['text/plain'],'Thedoodadwascreatedsohereisahaiku.');
//Heretheresponsebodywillbesetto
//thequerystring-encodedresultof
//FormModel.forClient({some:'data'})
//becausethestatuscodedefaultsto200.
router.patch(function(req,res){
//...
res.send({some:'data'});
})
.response(FormModel,['application/x-www-form-urlencoded'],'OMG.');
//Inthiscasetheresponsebodywillbesetto
//SomeModel.forClient({some:'data'})because
//thestatuscodehasbeensetto201before.
router.put(function(req,res){
//...
res.status(201);
res.send({some:'data'});
})
.response(201,SomeModel,'Somethingamazinghappened.');
error
Endpoints
199
endpoint.error(status,[description]):this
Documentsanerrorstatusfortheendpoint.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethiserrordescriptionunlessoverridden.Iftheendpointisamiddleware,thismethodhasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
status:number|string
HTTPstatuscodefortheerror(e.g.400for"badrequest").Ifastringisprovidedinsteadofanumericstatuscodeitwillbeusedtolookupanumericstatuscodeusingthestatusesmodule.
description:string(optional)
Ahuman-readablestringthatbrieflydescribestheerrorconditionandwillbeshownintheendpoint'sdetaileddocumentation.
Returnstheendpoint.
Examples
router.get(function(req,res){
//...
res.throw(403,'Validationerroratx.y.z');
})
.error(403,'Indicatesthatavalidationhasfailed.');
summaryendpoint.summary(summary):this
Addsashortdescriptiontotheendpoint'sAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethissummaryunlessoverridden.Iftheendpointisamiddleware,thismethodhasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
summary:string
Ahuman-readablestringthatbrieflydescribestheendpointandwillappearnexttotheendpoint'spathinthedocumentation.
Returnstheendpoint.
Examples
router.get(/*...*/)
.summary('Listalldiscombobulateddoodads')
descriptionendpoint.description(description):this
Addsalongdescriptiontotheendpoint'sAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisdescriptionunlessoverridden.Iftheendpointisamiddleware,thismethodhasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
Endpoints
200
description:string
Ahuman-readablestringthatdescribestheendpointindetailandwillbeshownintheendpoint'sdetaileddocumentation.
Returnstheendpoint.
Examples
//The"dedent"libraryhelpsformatting
//multi-linestringsbyadjustingindentation
//andremovingleadingandtrailingblanklines
constdd=require('dedent');
router.post(/*...*/)
.description(dd`
Thisroutediscombobulatesthedoodadsby
frobnicatingthemoxieoftherequestbody.
`)
deprecatedendpoint.deprecated([deprecated]):this
Markstheendpointasdeprecated.
Iftheendpointisachildrouter,allroutesofthatrouterwillalsobemarkedasdeprecated.Iftheendpointisamiddleware,thismethodhasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
deprecated:boolean(Default:true)
Whethertheendpointshouldbemarkedasdeprecated.Ifsettofalsetheendpointwillbeexplicitlymarkedasnotdeprecated.
Returnstheendpoint.
Examples
router.get(/*...*/)
.deprecated();
tagendpoint.tag(...tags):this
MarkstheendpointwiththegiventagsthatwillbeusedtogrouprelatedroutesinthegeneratedAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillalsobemarkedwiththetags.Iftheendpointisamiddleware,thismethodhasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
tags:string
Oneormorestringsthatwillbeusedtogrouptheendpoint'sroutes.
Returnstheendpoint.
Examples
router.get(/*...*/)
.tag('auth','restricted');
Endpoints
201
Endpoints
202
MiddlewareMiddlewareinFoxxreferstofunctionsthatcanbemountedlikeroutesandcanmanipulatetherequestandresponseobjectsbeforeandaftertherouteitselfisinvoked.Theycanalsobeusedtocontrolaccessortoprovidecommonlogiclikeloggingetc.Unlikeroutes,middlewareismountedwiththeusemethodlikearouter.
Insteadofafunctiontheusemethodcanalsoacceptanobjectwitharegisterfunctionthatwilltakeaparameterendpoint,themiddlewarewillbemountedatandreturnstheactualmiddlewarefunction.Thisallowsmanipulatingtheendpointbeforecreatingthemiddleware(e.g.todocumentheaders,requestbodies,pathparametersorqueryparameters).
Examples
RestrictaccesstoArangoDB-authenticatedusers:
module.context.use(function(req,res,next){
if(!req.arangoUser){
res.throw(401,'NotauthenticatedwithArangoDB');
}
next();
});
Anytruthyargumentpassedtothenextfunctionwillbethrownasanerror:
module.context.use(function(req,res,next){
leterr=null;
if(!req.arangoUser){
err=newError('Thisshouldneverhappen');
}
next(err);//throwsiftheerrorwasset
})
Trivialloggingmiddleware:
module.context.use(function(req,res,next){
conststart=Date.now();
try{
next();
}finally{
console.log(`Handledrequestin${Date.now()-start}ms`);
}
});
Morecomplexexampleforheader-basedsessions:
constsessions=module.context.collection('sessions');
module.context.use({
register(endpoint){
endpoint.header('x-session-id',joi.string().optional(),'ThesessionID.');
returnfunction(req,res,next){
constsid=req.get('x-session-id');
if(sid){
try{
req.session=sessions.document(sid);
}catch(e){
deletereq.headers['x-session-id'];
}
}
next();
if(req.session){
if(req.session._rev){
sessions.replace(req.session,req.session);
res.set('x-session-id',req.session._key);
}else{
constmeta=sessions.save(req.session);
res.set('x-session-id',meta._key);
}
Middleware
203
}
};
}
});
Middleware
204
RequestobjectsTherequestobjectspecifiesthefollowingproperties:
arangoUser:string|null
TheauthenticatedArangoDBusernameusedtomaketherequest.ThisvalueisonlysetifauthenticationisenabledinArangoDBandtherequestsetanauthorizationheaderArangoDBwasabletoverify.YouarestronglyencouragedtoimplementyourownauthenticationlogicforyourownservicesbutthispropertycanbeusefulifyouneedtointegratewithArangoDB'sownauthenticationmechanisms.
arangoVersion:number
Thenumericvalueofthex-arango-versionheaderorthenumericversionoftheArangoDBserver(e.g.30102forversion3.1.2)ifnovalidheaderwasprovided.
baseUrl:string
Root-relativebaseURLoftheservice,i.e.theprefix"/_db/"followedbythevalueofdatabase.
body:any
Theprocessedandvalidatedrequestbodyforthecurrentroute.Ifnobodyhasbeendefinedforthecurrentroute,thevaluewillbeidenticaltorawBody.
FordetailsonhowrequestbodiescanbeprocessedandvalidatedbyFoxxseethebodymethodoftheendpointobject.
context:Context
Theservicecontextinwhichtherouterwasmounted(ratherthanthecontextinwhichtheroutewasdefined).
database:string
Thenameofthedatabaseinwhichtherequestisbeinghandled,e.g."_system".
headers:object
Therawheadersobject.
FordetailsonhowrequestheaderscanbevalidatedbyFoxxseetheheadermethodoftheendpointobject.
hostname:string
Thehostname(domainname)indicatedintherequestheaders.
Defaultstothehostnameportion(i.e.excludingtheport)oftheHostheaderandfallsbacktothelisteningaddressoftheserver.
method:string
TheHTTPverbusedtomaketherequest,e.g."GET".
originalUrl:string
Root-relativeURLoftherequest,i.e.pathfollowedbytherawqueryparameters,ifany.
path:string
Database-relativepathoftherequestURL(notincludingthequeryparameters).
pathParams:object
Anobjectmappingthenamesofpathparametersofthecurrentroutetotheirvalidatedvalues.
FordetailsonhowpathparameterscanbevalidatedbyFoxxseethepathParammethodoftheendpointobject.
port:number
Theportindicatedintherequestheaders.
Request
205
Defaultstotheportportion(i.e.excludingthehostname)oftheHostheaderandfallsbacktothelisteningportortheappropriatedefaultport(443forHTTPSor80forHTTP,dependingonsecure)iftheheaderonlyindicatesahostname.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettotheportportionoftheX-Forwarded-Hostheader(orapproriatedefaultport)ifpresent.
protocol:string
Theprotocolusedfortherequest.
Defaultsto"https"or"http"dependingonwhetherArangoDBisconfiguredtouseSSLornot.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothevalueoftheX-Forwarded-Protoheaderifpresent.
queryParams:object
Anobjectmappingthenamesofqueryparametersofthecurrentroutetotheirvalidatedvalues.
FordetailsonhowqueryparameterscanbevalidatedbyFoxxseethequeryParammethodoftheendpointobject.
rawBody:Buffer
Theraw,unparsed,unvalidatedrequestbodyasabuffer.
remoteAddress:string
TheIPoftheclientthatmadetherequest.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothefirstIPlistedintheX-Forwarded-Forheaderifpresent.
remoteAddresses:Array<string>
AlistcontainingtheIPaddressesusedtomaketherequest.
DefaultstothevalueofremoteAddresswrappedinanarray.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothelistofIPsspecifiedintheX-Forwarded-Forheaderifpresent.
remotePort:number
Thelisteningportoftheclientthatmadetherequest.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettotheportspecifiedintheX-Forwarded-Portheaderifpresent.
secure:boolean
Whethertherequestwasmadeoverasecureconnection(i.e.HTTPS).
Thisissettofalsewhenprotocolis"http"andtruewhenprotocolis"https".
suffix:string
Thetrailingpathrelativetothecurrentrouteifthecurrentrouteendsinawildcard(e.g./something/*).
Note:StartingwithArangoDB3.2ispassedintotheserviceas-is,i.e.percentageescapesequenceslike%2Fwillnolongerbeunescaped.Alsonotethatthesuffixmaycontainpathsegmentslike..whichmayhavespecialmeaningifthesuffixisusedtobuildfilesystempaths.
trustProxy:boolean
Indicateswhethertherequestwasmadeusingatrustedproxy.Iftheoriginserver'saddresswasspecifiedintheArangoDBconfigurationusing--frontend.trusted-proxyortheservice'strustProxysettingisenabled,thiswillbetrue,otherwiseitwillbefalse.
url:string
TheURLoftherequest.
Request
206
xhr:boolean
WhethertherequestindicatesitwasmadewithinabrowserusingAJAX.
ThisissettotrueiftheX-Requested-Withheaderispresentandisacase-insensitivematchforthevalue"xmlhttprequest".
NotethatthisvaluedoesnotguaranteewhethertherequestwasmadefrominsideabrowserorwhetherAJAXwasusedandismerelyaconventionestablishedbyJavaScriptframeworkslikejQuery.
acceptsreq.accepts(types):string|false
req.accepts(...types):string|false
req.acceptsCharsets(charsets):string|false
req.acceptsCharsets(...charsets):string|false
req.acceptsEncodings(encodings):string|false
req.acceptsEncodings(...encodings):string|false
req.acceptsLanguages(languages):string|false
req.acceptsLanguages(...languages):string|false
Thesemethodswrapthecorrespondingcontentnegotiationmethodsoftheacceptsmoduleforthecurrentrequest.
Examples
if(req.accepts(['json','html'])==='html'){
//ClientexplicitlyprefersHTMLoverJSON
res.write('<h1>ClientprefersHTML</h1>');
}else{
//OtherwisejustsendJSON
res.json({success:true});
}
cookiereq.cookie(name,options):string|null
Getsthevalueofacookiebyname.
Arguments
name:string
Nameofthecookie.
options:object(optional)
Anobjectwithanyofthefollowingproperties:
secret:string(optional)
Secretthatwasusedtosignthecookie.
Ifasecretisspecified,thecookie'ssignatureisexpectedtobepresentinasecondcookiewiththesamenameandthesuffix.sig.Otherwisethesignature(ifpresent)willbeignored.
algorithm:string(Default:"sha256")
Algorithmthatwasusedtosignthecookie.
Ifastringispassedinsteadofanoptionsobjectitwillbeinterpretedasthesecretoption.
Returnsthevalueofthecookieornullifthecookieisnotsetoritssignatureisinvalid.
Request
207
get/headerreq.get(name):string
req.header(name):string
Getsthevalueofaheaderbyname.Youcanvalidaterequestheadersusingtheheadermethodoftheendpoint.
Arguments
name:string
Nameoftheheader.
Returnstheheadervalue.
isreq.is(types):string
req.is(...types):string
Thismethodwrapsthe(requestbody)contenttypedetectionmethodofthetype-ismoduleforthecurrentrequest.
Examples
consttype=req.is('html','application/xml','application/*+xml');
if(type===false){//nomatch
handleDefault(req.rawBody);
}elseif(type==='html'){
handleHtml(req.rawBody);
}else{//isXML
handleXml(req.rawBody);
}
jsonreq.json():any
AttemptstoparsetherawrequestbodyasJSONandreturnstheresult.
Itisgenerallymoreusefultodefinearequestbodyontheendpointandusethereq.bodypropertyinstead.
Returnsundefinediftherequestbodyisempty.MaythrowaSyntaxErrorifthebodycouldnotbeparsed.
makeAbsolutereq.makeAbsolute(path,[query]):string
Resolvesthegivenpathrelativetothereq.context.service'smountpathtoafullURL.
Arguments
path:string
Thepathtoresovle.
query:string|object
AstringorobjectwithqueryparameterstoaddtotheURL.
ReturnstheformattedabsoluteURL.
paramsreq.param(name):any
Request
208
Arguments
Looksupaparameterbyname,preferringpathParamsoverqueryParams.
It'sprobablybetterstyletousethereq.pathParamsorreq.queryParamsobjectsdirectly.
name:string
Nameoftheparameter.
Returnsthe(validated)valueoftheparameter.
rangereq.range([size]):Ranges|number
Thismethodwrapstherangeheaderparsingmethodoftherange-parsermoduleforthecurrentrequest.
Arguments
size:number(Default:Infinity)
Lengthofthesatisfiablerange(e.g.numberofbytesinthefullresponse).Ifpresent,rangesexceedingthesizewillbeconsideredunsatisfiable.
ReturnsundefinediftheRangeheaderisabsent,-2iftheheaderispresentbutmalformed,-1iftherangeisinvalid(e.g.startoffsetislargerthanendoffset)orunsatisfiableforthegivensize.
Otherwisereturnsanarrayofobjectswiththepropertiesstartandendvaluesforeachrange.Thearrayhasanadditionalpropertytypeindicatingtherequestrangetype.
Examples
console.log(req.headers.range);//"bytes=40-80"
constranges=req.range(100);
console.log(ranges);//[{start:40,end:80}]
console.log(ranges.type);//"bytes"
reversereq.reverse(name,[params]):string
LooksuptheURLofanamedrouteforthegivenparameters.
Arguments
name:string
Nameoftheroutetolookup.
params:object(optional)
Anobjectcontainingvaluesforthe(pathorquery)parametersoftheroute.
ReturnstheURLoftherouteforthegivenparameters.
Examples
router.get('/items/:id',function(req,res){
/*...*/
},'getItemById');
router.post('/items',function(req,res){
//...
consturl=req.reverse('getItemById',{id:createdItem._key});
res.set('location',req.makeAbsolute(url));
});
Request
209
Request
210
ResponseobjectsTheresponseobjectspecifiesthefollowingproperties:
body:Buffer|string
Responsebodyasastringorbuffer.Canbesetdirectlyorusingsomeoftheresponsemethods.
context:Context
Theservicecontextinwhichtherouterwasmounted(ratherthanthecontextinwhichtheroutewasdefined).
headers:object
Therawheadersobject.
statusCode:number
Statuscodeoftheresponse.Defaultsto200(bodysetandnotanemptystringorbuffer)or204(otherwise)ifnotchangedfromundefined.
attachmentres.attachment([filename]):this
Setsthecontent-dispositionheadertoindicatetheresponseisadownloadablefilewiththegivenname.
Note:Thisdoesnotactuallymodifytheresponsebodyoraccessthefilesystem.TosendafilefromthefilesystemseethedownloadorsendFilemethods.
Arguments
filename:string(optional)
Nameofthedownloadablefileintheresponsebody.
Ifpresent,theextensionofthefilenamewillbeusedtosettheresponsecontent-typeifithasnotyetbeenset.
Returnstheresponseobject.
cookieres.cookie(name,value,[options]):this
Setsacookiewiththegivenname.
Arguments
name:string
Nameofthecookie.
value:string
Valueofthecookie.
options:object(optional)
Anobjectwithanyofthefollowingproperties:
ttl:number(optional)
Timetoliveofthecookieinseconds.
algorithm:string(Default:"sha256")
Algorithmthatwillbeusedtosignthecookie.
Response
211
secret:string(optional)
Secretthatwillbeusedtosignthecookie.
Ifasecretisspecified,thecookie'ssignaturewillbestoredinasecondcookiewiththesameoptions,thesamenameandthesuffix.sig.Otherwisenosignaturewillbeadded.
path:string(optional)
Pathforwhichthecookieshouldbeissued.
domain:string(optional)
Domainforwhichthecookieshouldbeissued.
secure:boolean(Default:false)
Whetherthecookieshouldbemarkedassecure(i.e.HTTPS/SSL-only).
httpOnly:boolean(Default:false)
WhetherthecookieshouldbemarkedasHTTP-only(ratherthanalsoexposingittoclient-sidecode).
Ifastringispassedinsteadofanoptionsobjectitwillbeinterpretedasthesecretoption.
Ifanumberispassedinsteadofanoptionsobjectitwillbeinterpretedasthettloption.
Returnstheresponseobject.
downloadres.download(path,[filename]):this
Theequivalentofcallingres.attachment(filename).sendFile(path).
Arguments
path:string
Pathtothefileonthelocalfilesystemtobesentastheresponsebody.
filename:string(optional)
Filenametoindicateinthecontent-dispositionheader.
Ifomittedthepathwillbeusedinstead.
Returnstheresponseobject.
getHeaderres.getHeader(name):string
Getsthevalueoftheheaderwiththegivenname.
Arguments
name:string
Nameoftheheadertoget.
Returnsthevalueoftheheaderorundefined.
jsonres.json(data):this
SetstheresponsebodytotheJSONstringvalueofthegivendata.
Response
212
Arguments
data:any
Thedatatobeusedastheresponsebody.
Returnstheresponseobject.
redirectres.redirect([status],path):this
Redirectstheresponsebysettingtheresponselocationheaderandstatuscode.
Arguments
status:number|string(optional)
Responsestatuscodetoset.
Ifthestatuscodeisthestringvalue"permanent"itwillbetreatedasthevalue301.
Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.
Ifthestatuscodeisomittedbuttheresponsestatushasnotalreadybeenset,theresponsestatuswillbesetto302.
path:string
URLtosetthelocationheaderto.
Returnstheresponseobject.
removeHeaderres.removeHeader(name):this
Removestheheaderwiththegivennamefromtheresponse.
Arguments
name:string
Nameoftheheadertoremove.
Returnstheresponseobject.
sendres.send(data,[type]):this
Setstheresponsebodytothegivendatawithrespecttotheresponsedefinitionfortheresponse'scurrentstatuscode.
Arguments
data:any
Thedatatobeusedastheresponsebody.Willbeconvertedaccordingtheresponsedefinitionfortheresponse'scurrentstatuscode(or200)inthefollowingway:
IfthedataisanArangoDBresultset,itwillbeconvertedtoanarrayfirst.
IftheresponsedefinitionspecifiesamodelwithaforClientmethod,thatmethodwillbeappliedtothedatafirst.Ifthedataisanarrayandtheresponsedefinitionhasthemultipleflagset,themethodwillbeappliedtoeachentryindividuallyinstead.
Finallythedatawillbeprocessedbytheresponsetypehandlertoconvertheresponsebodytoastringorbuffer.
type:string(Default:"auto")
Response
213
Content-typeoftheresponsebody.
Ifsetto"auto"thefirstMIMEtypespecifiedintheresponsedefinitionfortheresponse'scurrentstatuscode(or200)willbeusedinstead.
Ifsetto"auto"andnoresponsedefinitionexists,theMIMEtypewillbedeterminedthefollowingway:
IfthedataisabuffertheMIMEtypewillbesettobinary(application/octet-stream).
IfthedataisanobjecttheMIMEtypewillbesettoJSONandthedatawillbeconvertedtoaJSONstring.
OtherwisetheMIMEtypewillbesettoHTMLandthedatawillbeconvertedtoastring.
Returnstheresponseobject.
sendFileres.sendFile(path,[options]):this
Sendsafilefromthelocalfilesystemastheresponsebody.
Arguments
path:string
Pathtothefileonthelocalfilesystemtobesentastheresponsebody.
Ifnocontent-typeheaderhasbeensetyet,theextensionofthefilenamewillbeusedtosetthevalueofthatheader.
options:object(optional)
Anobjectwithanyofthefollowingproperties:
lastModified:boolean(optional)
Ifsettotrueorifnolast-modifiedheaderhasbeensetyetandthevalueisnotsettofalsethelast-modifiedheaderwillbesettothemodificationdateofthefileinmilliseconds.
Returnstheresponseobject.
Examples
//Sendthefile"favicon.ico"fromthisservice'sfolder
res.sendFile(module.context.fileName('favicon.ico'));
sendStatusres.sendStatus(status):this
Sendsaplaintextresponseforthegivenstatuscode.Theresponsestatuswillbesettothegivenstatuscode,theresponsebodywillbesettothestatusmessagecorrespondingtothatstatuscode.
Arguments
status:number|string
Responsestatuscodetoset.
Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.
Returnstheresponseobject.
setHeader/setres.setHeader(name,value):this
res.set(name,value):this
Response
214
res.set(headers):this
Setsthevalueoftheheaderwiththegivenname.
Arguments
name:string
Nameoftheheadertoset.
value:string
Valuetosettheheaderto.
headers:object
Headerobjectmappingheadernamestovalues.
Returnstheresponseobject.
statusres.status(status):this
Setstheresponsestatustothegivenstatuscode.
Arguments
status:number|string
Responsestatuscodetoset.
Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.
Returnstheresponseobject.
throwres.throw(status,[reason],[options]):void
ThrowsanHTTPexceptionforthegivenstatus,whichwillbehandledbyFoxxtoservetheappropriateJSONerrorresponse.
Arguments
status:number|string
Responsestatuscodetoset.
Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.
Ifthestatuscodeisinthe500-range(500-599),itsstacktracewillalwaysbeloggedasifitwereanunhandledexception.
Ifdevelopmentmodeisenabled,theerror'sstacktracewillbeloggedasawarningifthestatuscodeisinthe400-range(400-499)orasaregularmessageotherwise.
reason:string(optional)
Messagefortheexception.
Ifomitted,thestatusmessagecorrespondingtothestatuscodewillbeusedinstead.
options:object(optional)
Anobjectwithanyofthefollowingproperties:
cause:Error(optional)
Causeoftheexceptionthatwillbeloggedaspartoftheerror'sstacktrace(recursively,iftheexceptionalsohasacausepropertyandsoon).
Response
215
extra:object(optional)
AdditionalpropertiesthatwillbeaddedtotheerrorresponsebodygeneratedbyFoxx.
Ifdevelopmentmodeisenabled,anexceptionpropertywillbeaddedtothisvaluecontainingtheerrormessageandastacktracepropertywillbeaddedcontaininganarraywitheachlineoftheerror'sstacktrace.
Ifanerrorispassedinsteadofanoptionsobjectitwillbeinterpretedasthecauseoption.Ifnoreasonwasprovidedtheerror'smessagewillbeusedasthereasoninstead.
Returnsnothing.
typeres.type([type]):string
Setstheresponsecontent-typetothegiventypeifprovidedorreturnsthepreviouslysetcontent-type.
Arguments
type:string(optional)
Content-typeoftheresponsebody.
Unlikeres.set('content-type',type)fileextensionscanbeprovidedasvaluesandwillbetranslatedtothecorrespondingMIMEtype(e.g.jsonbecomesapplication/json).
Returnsthecontent-typeoftheresponsebody.
varyres.vary(names):this
res.vary(...names):this
Thismethodwrapsthevaryheadermanipulationmethodofthevarymoduleforthecurrentresponse.
Thegivennameswillbeaddedtotheresponse'svaryheaderifnotalreadypresent.
Returnstheresponseobject.
Examples
res.vary('user-agent');
res.vary('cookie');
res.vary('cookie');//duplicateswillbeignored
//--or--
res.vary('user-agent','cookie');
//--or--
res.vary(['user-agent','cookie']);
writeres.write(data):this
Appendsthegivendatatotheresponsebody.
Arguments
data:string|Buffer
Datatoappend.
Response
216
Ifthedataisabuffertheresponsebodywillbeconvertedtoabufferfirst.
Iftheresponsebodyisabufferthedatawillbeconvertedtoabufferfirst.
IfthedataisanobjectitwillbeconvertedtoaJSONstringfirst.
Ifthedataisanyothernon-stringvalueitwillbeconvertedtoastringfirst.
Returnstheresponseobject.
Response
217
UsingGraphQLinFoxxconstcreateGraphQLRouter=require('@arangodb/foxx/graphql');
Foxxbundlesthegraphql-syncmodule,whichisasynchronouswrapperfortheofficialJavaScriptGraphQLreferenceimplementation,toallowwritingGraphQLschemasdirectlyinsideFoxx.
Additionallythe@arangodb/foxx/graphqlletsyoucreateroutersforservingGraphQLrequests,whichcloselymimicksthebehaviouroftheexpress-graphqlmodule.
Formoreinformationongraphql-syncseethegraphql-jsAPIreference(notethatgraphql-syncalwaysusesrawvaluesinsteadofwrappingtheminpromises).
ForanexampleofaGraphQLschemainFoxxthatresolvesfieldsusingthedatabaseseetheGraphQLexampleservice(alsoavailablefromtheFoxxstore).
Examples
constgraphql=require('graphql-sync');
constgraphqlSchema=newgraphql.GraphQLSchema({
//...
});
//Mountingagraphqlendpointdirectlyinaservice:
module.context.use('/graphql',createGraphQLRouter({
schema:graphqlSchema,
graphiql:true
}));
//Orattheservice'srootURL:
module.context.use(createGraphQLRouter({
schema:graphqlSchema,
graphiql:true
}));
//Orinsideanexistingrouter:
router.get('/hello',function(req,res){
res.write('Helloworld!');
});
router.use('/graphql',createGraphQLRouter({
schema:graphqlSchema,
graphiql:true
}));
Note:ArangoDBaimsforstabilitywhichmeansbundleddependencieswillgenerallynotbeupdatedasquicklyastheirmaintainersmakeupdatesavailableonGitHuborNPM.StartingwithArangoDB3.2,ifyouwanttouseanewerreleaseofgraphql-syncthantheonebundledwithyourtargetversionofArangoDB,youcanprovideyourownversionofthelibrarybypassingitviathegraphqloption:
constgraphql=require('graphql-sync');
constgraphqlSchema=newgraphql.Schema({
//...
});
module.context.use(createGraphQLRouter({
schema:graphqlSchema,
graphiql:true,
graphql:graphql
}))
ThismakessureFoxxusesthegraphql-syncmodulebundledinyourservice'snode_modulesfolder(ifavailable)insteadofthebuilt-inversion.Ifyoufindgraphql-syncitselflaggingbehindtheofficialgraphqlmodule,consideropeninganissueonitsGitHubrepository.
CreatingaroutercreateGraphQLRouter(options):Router
UsingGraphQL
218
ThisreturnsanewrouterobjectwithPOSTandGETroutesforservingGraphQLrequests.
Arguments
options:object
Anobjectwithanyofthefollowingproperties:
schema:GraphQLSchema
AGraphQLSchemaobjectfromgraphql-sync.
context:any(optional)
TheGraphQLcontextthatwillbepassedtothegraphql()functionfromgraphql-synctohandleGraphQLqueries.
rootValue:object(optional)
TheGraphQLrootvaluethatwillbepassedtothegraphql()functionfromgraphql-synctohandleGraphQLqueries.
pretty:boolean(Default:false)
Iftrue,JSONresponseswillbepretty-printed.
formatError:Function(optional)
Afunctionthatwillbeusedtoformaterrorsproducedbygraphql-sync.Ifomitted,theformatErrorfunctionfromgraphql-syncwillbeusedinstead.
validationRules:Array<any>(optional)
AdditionalvalidationrulesqueriesmustsatisfyinadditiontothosedefinedintheGraphQLspec.
graphiql:boolean(Default:false)
Iftrue,theGraphiQLexplorerwillbeservedwhenloadeddirectlyfromabrowser.
graphql:object(optional)
Ifyouneedtouseyourowncopyofthegraphql-syncmoduleinsteadoftheonebundledwithArangoDB,hereyoucanpassitindirectly.
IfaGraphQLSchemaobjectispassedinsteadofanoptionsobjectitwillbeinterpretedastheschemaoption.
Generatedroutes
TherouterhandlesGETandPOSTrequeststoitsrootpathandacceptsthefollowingparameters,whichcanbeprovidedeitherasqueryparametersorasthePOSTrequestbody:
query:string
AGraphQLquerythatwillbeexecuted.
variables:object|string(optional)
AnobjectorastringcontainingaJSONobjectwithruntimevaluestouseforanyGraphQLqueryvariables.
operationName:string(optional)
Iftheprovidedquerycontainsmultiplenamedoperations,thisspecifieswhichoperationshouldbeexecuted.
raw:boolean(Default:false)
ForcesaJSONresponseevenifgraphiqlisenabledandtherequestwasmadeusingabrowser.
ThePOSTrequestbodycanbeprovidedasJSONorasquerystringusingapplication/x-www-form-urlencoded.Arequestbodypassedasapplication/graphqlwillbeinterpretedasthequeryparameter.
UsingGraphQL
219
SessionMiddlewareconstsessionMiddleware=require('@arangodb/foxx/sessions');
ThesessionmiddlewareaddsthesessionandsessionStoragepropertiestotherequestobjectanddealswithserializinganddeserializingthesessionaswellasextractingsessionidentifiersfromincomingrequestsandinjectingthemintooutgoingresponses.
Examples
//Createasessionmiddleware
constsessions=sessionsMiddleware({
storage:module.context.collection('sessions'),
transport:['header','cookie']
});
//Firstenablethemiddlewareforthisservice
module.context.use(sessions);
//Nowmounttheroutersthatusethesession
constrouter=createRouter();
module.context.use(router);
router.get('/',function(req,res){
res.send(`Hello${req.session.uid||'anonymous'}!`);
},'hello');
router.post('/login',function(req,res){
req.session.uid=req.body;
req.sessionStorage.save(req.session);
res.redirect(req.reverse('hello'));
});
.body(['text/plain'],'Username');
CreatingasessionmiddlewaresessionMiddleware(options):Middleware
Createsasessionmiddleware.
Arguments
options:Object
Anobjectwiththefollowingproperties:
storage:Storage
Storagethatwillbeusedtopersistthesessions.
ThestorageisalsoexposedasthesessionStorageonallrequestobjectsandasthestoragepropertyofthemiddleware.
IfastringorcollectionispassedinsteadofaStorage,itwillbeusedtocreateaCollectionStorage.
transport:Transport|Array<Transport>
Transportorarrayoftransportsthatwillbeusedtoextractthesessionidentifiersfromincomingrequestsandinjectthemintooutgoingresponses.Whenattemptingtoextractasessionidentifier,thetransportswillbeusedintheorderspecifieduntilamatchisfound.Wheninjecting(orclearing)sessionidentifiers,alltransportswillbeinvoked.
Thetransportsarealsoexposedasthetransportpropertyofthemiddleware.
Ifthestring"cookie"ispassedinsteadofaTransport,theCookieTransportwillbeusedwiththedefaultsettingsinstead.
Ifthestring"header"ispassedinsteadofaTransport,theHeaderTransportwillbeusedwiththedefaultsettingsinstead.
autoCreate:boolean(Default:true)
Ifenabledthesessionstorage'snewmethodwillbeinvokedtocreateanemptysessionwheneverthetransportfailedtoreturnasessionfortheincomingrequest.Otherwisethesessionwillbeinitializedasnull.
Sessionsmiddleware
220
Returnsthesessionmiddleware.
Sessionsmiddleware
221
SessionStoragesSessionstoragesareusedbythesessionsmiddlewaretopersistsessionsacrossrequests.SessionstoragesmustimplementthefromClientandforClientmethodsandcanoptionallyimplementthenewmethod.
Thebuilt-insessionstoragesgenerallyprovidethefollowingattributes:
uid:string(Default:null)
Auniqueidentifierindicatingtheactiveuser.
created:number(Default:Date.now())
Thenumerictimestampofwhenthesessionwascreated.
data:any(Default:null)
Arbitrarydatatopersistedinthesession.
newstorage.new():Session
Generatesanewsessionobjectrepresentinganemptysession.Theemptysessionobjectshouldnotbepersistedunlessnecessary.Thereturnvaluewillbeexposedbythemiddlewareasthesessionpropertyoftherequestobjectifnosessionidentifierwasreturnedbythesessiontransportsandauto-creationisnotexplicitlydisabledinthesessionmiddleware.
Examples
new(){
return{
uid:null,
created:Date.now(),
data:null
};
}
fromClientstorage.fromClient(sid):Session|null
Resolvesordeserializesasessionidentifiertoasessionobject.
Arguments
sid:string
Sessionidentifiertoresolveordeserialize.
Returnsasessionobjectrepresentingthesessionwiththegivensessionidentifierthatwillbeexposedbythemiddlewareasthesessionpropertyoftherequestobject.Thismethodwillonlybecalledifanyofthesessiontransportsreturnedasessionidentifier.Ifthesessionidentifierisinvalidorexpired,themethodshouldreturnanullvaluetoindicatenomatchingsession.
Examples
fromClient(sid){
returndb._collection('sessions').firstExample({_key:sid});
}
forClientstorage.forClient(session):string|null
Sessionstorages
222
Derivesasessionidentifierfromthegivensessionobject.
Arguments
session:Session
Sessiontoderiveasessionidentifierfrom.
Returnsasessionidentifierforthesessionrepresentedbythegivensessionobject.Thismethodwillbecalledwiththesessionpropertyoftherequestobjectunlessthatpropertyisempty(e.g.null).
Examples
forClient(session){
if(!session._key){
constmeta=db._collection('sessions').save(session);
returnmeta._key;
}
db._collection('sessions').replace(session._key,session);
returnsession._key;
}
Sessionstorages
223
CollectionSessionStorageconstcollectionStorage=require('@arangodb/foxx/sessions/storages/collection');
Thecollectionsessionstoragepersistssessionstoacollectioninthedatabase.
CreatingastoragecollectionStorage(options):Storage
CreatesaStoragethatcanbeusedinthesessionsmiddleware.
Arguments
options:Object
Anobjectwiththefollowingproperties:
collection:ArangoCollection
Thecollectionthatshouldbeusedtopersistthesessions.Ifastringispassedinsteadofacollectionitisassumedtobethefullyqualifiednameofacollectioninthecurrentdatabase.
ttl:number(Default:60*60)
Thetimeinsecondssincethelastupdateuntilasessionwillbeconsideredexpired.
pruneExpired:boolean(Default:false)
Whetherexpiredsessionsshouldberemovedfromthecollectionwhentheyareaccessedinsteadofsimplybeingignored.
autoUpdate:boolean(Default:true)
Whethersessionsshouldbeupdatedinthecollectioneverytimetheyareaccessedtokeepthemfromexpiring.Disablingthisoptionwillimproveperformancebutmeansyouwillhavetotakecareofkeepingyoursessionsaliveyourself.
Ifastringorcollectionispassedinsteadofanoptionsobject,itwillbeinterpretedasthecollectionoption.
prunestorage.prune():Array<string>
Removesallexpiredsessionsfromthecollection.ThismethodshouldbecalledevenifthepruneExpiredoptionisenabledtocleanupabandonedsessions.
Returnsanarrayofthekeysofallsessionsthatwereremoved.
savestorage.save(session):Session
Saves(replaces)thegivensessionobjectinthecollection.Thismethodneedstobeinvokedexplicitlyaftermakingchangestothesessionorthechangeswillnotbepersisted.Assignsanew_keytothesessionifitpreviouslydidnothaveone.
Arguments
session:Session
Asessionobject.
Returnsthemodifiedsession.
clear
Sessionstorages
224
storage.clear(session):boolean
Removesthesessionfromthecollection.Hasnoeffectifthesessionwasalreadyremovedorhasnotyetbeensavedtothecollection(i.e.hasno_key).
Arguments
session:Session
Asessionobject.
Returnstrueifthesessionwasremovedorfalseifithadnoeffect.
Sessionstorages
225
JWTSessionStorageconstjwtStorage=require('@arangodb/foxx/sessions/storages/jwt');
TheJWTsessionstorageconvertssessionstoandfromJSONWebTokens.
Examples
//PassinasecuresecretfromtheFoxxconfiguration
constsecret=module.context.configuration.jwtSecret;
constsessions=sessionsMiddleware({
storage:jwtStorage(secret),
transport:'header'
});
module.context.use(sessions);
CreatingastoragejwtStorage(options):Storage
CreatesaStoragethatcanbeusedinthesessionsmiddleware.
Note:whilethe"none"algorithm(i.e.nosignature)issupportedthisdummyalgorithmprovidesnosecurityandallowsclientstomakearbitrarymodificationstothepayloadandshouldnotbeusedunlessyouarecertainyouspecificallyneedit.
Arguments
options:Object
Anobjectwiththefollowingproperties:
algorithm:string(Default:"HS512")
Thealgorithmtouseforsigningthetoken.
Supportedvalues:
"HS256"(HMAC-SHA256)"HS384"(HMAC-SHA384)"HS512"(HMAC-SHA512)"none"(nosignature)
secret:string
Thesecrettouseforsigningthetoken.
Thisfieldisforbiddenwhenusingthe"none"algorithmbutrequiredotherwise.
ttl:number(Default:3600)
Themaximumlifetimeofthetokeninseconds.Youmaywanttokeepthisshortasanewtokenisgeneratedoneveryrequestallowingclientstorefreshtokensautomatically.
verify:boolean(Default:true)
Ifsettofalsethesignaturewillnotbeverifiedbutstillgenerated(unlessusingthe"none"algorithm).
maxExp:number(Default:Infinity)
LargestvaluethatwillbeacceptedinanincomingJWTexp(expiration)field.
Ifastringispassedinsteadofanoptionsobjectitwillbeinterpretedasthesecretoption.
Sessionstorages
226
SessionTransportsSessiontransportsareusedbythesessionsmiddlewaretostoreandretrievesessionidentifiersinrequestsandresponses.Sessiontransportsmustimplementthegetand/orsetmethodsandcanoptionallyimplementtheclearmethod.
gettransport.get(request):string|null
Retrievesasessionidentifierfromarequestobject.
Ifpresentthismethodwillautomaticallybeinvokedforeachtransportuntilatransportreturnsasessionidentifier.
Arguments
request:Request
Requestobjecttoextractasessionidentifierfrom.
Returnsthesessionidentifierornullifthetransportcannotfindasessionidentifierintherequest.
Examples
get(req){
returnreq.get('x-session-id')||null;
}
settransport.set(response,sid):void
Attachesasessionidentifiertoaresponseobject.
Ifpresentthismethodwillautomaticallybeinvokedattheendofarequestregardlessofwhetherthesessionwasmodifiedornot.
Arguments
response:Response
Responseobjecttoattachasessionidentifierto.
sid:string
Sessionidentifiertoattachtotheresponse.
Returnsnothing.
Examples
set(res){
res.set('x-session-id',value);
}
cleartransport.clear(response):void
Attachesapayloadindicatingthatthesessionhasbeenclearedtotheresponseobject.Thiscanbeusedtoclearasessioncookiewhenthesessionhasbeendestroyed(e.g.duringlogout).
Ifpresentthismethodwillautomaticallybeinvokedinsteadofsetwhenthereq.sessionattributewasremovedbytheroutehandler.
Arguments
Sessiontransports
227
response:Response
Responseobjecttoremovethesessionidentifierfrom.
Returnsnothing.
Sessiontransports
228
CookieSessionTransportconstcookieTransport=require('@arangodb/foxx/sessions/transports/cookie');
Thecookietransportstoressessionidentifiersincookiesontherequestandresponseobject.
Examples
//PassinasecuresecretfromtheFoxxconfiguration
constsecret=module.context.configuration.cookieSecret;
constsessions=sessionsMiddleware({
storage:module.context.collection('sessions'),
transport:cookieTransport({
name:'FOXXSESSID',
ttl:60*60*24*7,//oneweekinseconds
algorithm:'sha256',
secret:secret
})
});
module.context.use(sessions);
CreatingatransportcookieTransport([options]):Transport
CreatesaTransportthatcanbeusedinthesessionsmiddleware.
Arguments
options:Object(optional)
Anobjectwiththefollowingproperties:
name:string(Default:"sid")
Thenameofthecookie.
ttl:number(optional)
Cookielifetimeinseconds.NotethatthisdoesnotaffectthestorageTTL(i.e.howlongthesessionitselfisconsideredvalid),justhowlongthecookieshouldbestoredbytheclient.
algorithm:string(optional)
Thealgorithmusedtosignandverifythecookie.Ifnoalgorithmisspecified,thecookiewillnotbesignedorverified.Seethecookiemethodontheresponseobject.
secret:string(optional)
Secrettouseforthesignedcookie.Willbeignoredifnoalgorithmisprovided.
path:string(optional)
Pathforwhichthecookieshouldbeissued.
domain:string(optional)
Domainforwhichthecookieshouldbeissued.
secure:boolean(Default:false)
Whetherthecookieshouldbemarkedassecure(i.e.HTTPS/SSL-only).
httpOnly:boolean(Default:false)
WhetherthecookieshouldbemarkedasHTTP-only(ratherthanalsoexposingittoclient-sidecode).
Ifastringispassedinsteadofanoptionsobject,itwillbeinterpretedasthenameoption.
Sessiontransports
229
Sessiontransports
230
HeaderSessionTransportconstheaderTransport=require('@arangodb/foxx/sessions/transports/header');
Theheadertransportstoressessionidentifiersinheadersontherequestandresponseobjects.
Examples
constsessions=sessionsMiddleware({
storage:module.context.collection('sessions'),
transport:headerTransport('X-FOXXSESSID')
});
module.context.use(sessions);
CreatingatransportheaderTransport([options]):Transport
CreatesaTransportthatcanbeusedinthesessionsmiddleware.
Arguments
options:Object(optional)
Anobjectwiththefollowingproperties:
name:string(Default:X-Session-Id)
Nameoftheheaderthatcontainsthesessionidentifier(notcasesensitive).
Ifastringispassedinsteadofanoptionsobject,itwillbeinterpretedasthenameoption.
Sessiontransports
231
StaticfileassetsThemostflexiblewaytoservefilesinyourFoxxserviceistosimplypassthemthroughinyourrouterusingthecontextobject'sfileNamemethodandtheresponseobject'ssendFilemethod:
router.get('/some/filename.png',function(req,res){
constfilePath=module.context.fileName('some-local-filename.png');
res.sendFile(filePath);
});
Whileallowingforgreatercontrolofhowthefileshouldbesenttotheclientandwhoshouldbeabletoaccessit,doingthisforallyourstaticassetscangettedious.
AlternativelyyoucanspecifyfileassetsthatshouldbeservedbyyourFoxxservicedirectlyintheservicemanifestusingthefilesattribute:
"files":{
"/some/filename.png":{
"path":"some-local-filename.png",
"type":"image/png",
"gzip":false
},
"/favicon.ico":"bookmark.ico",
"/static":"my-assets-folder"
}
Eachentryinthefilesattributecanrepresenteitherasinglefileoradirectory.Whenservingentiredirectories,thekeyactsasaprefixandrequeststothatprefixwillberesolvedwithinthegivendirectory.
Options
path:string
Therelativepathofthefileorfolderwithintheservice.
type:string(optional)
TheMIMEcontenttypeofthefile.Defaultstoanintelligentguessbasedonthefilename'sextension.
gzip:boolean(Default:false)
Ifsettotruethefilewillbeservedwithgzip-encodingifsupportedbytheclient.Thiscanbeusefulwhenservingtextfileslikeclient-sideJavaScript,CSSorHTML.
Ifastringisprovidedinsteadofanobject,itwillbeinterpretedasthepathoption.
Servingfiles
232
WritingtestsFoxxprovidesoutoftheboxsupportforrunningtestsagainstaninstalledserviceusingtheMochatestrunner.
TestfileshavefullaccesstotheservicecontextandallArangoDBAPIsbutlikescriptscannotdefineFoxxroutes.
Runningtests
Aninstalledservice'stestscanbeexecutedfromtheadministrativewebinterface:
1. Openthe"Services"tabofthewebinterface2. Clickontheinstalledservicetobetested3. Clickonthe"Settings"tab4. Clickontheflaskiconinthetopright5. Accepttheconfirmationdialog
Notethatrunningtestsinaproductiondatabaseisnotrecommendedandmayresultindatalossifthetestsaccessthedatabase.
Whenrunningaserviceindevelopmentmodespecialcareneedstobetakenasperformingrequeststotheservice'sownroutesaspartofthetestsuitesmayresultintestsbeingexecutedwhilethedatabaseisinaninconsistentstate,leadingtounexpectedbehaviour.
TestfilepathsInordertotellFoxxaboutfilescontainingtestsuites,oneormorepatternsneedtobespecifiedinthetestsoptionoftheservicemanifest:
{
"tests":[
"**/test_*.js",
"**/*_test.js"
]
}
Thesepatternscanbeeitherrelativefilepathsor"globstar"patternswhere
*matcheszeroormorecharactersinafilename**matcheszeroormorenesteddirectories
Forexample,giventhefollowingdirectorystructure:
++test/
|++a/
||+-a1.js
||+-a2.js
||+-test.js
|+-b.js
|+-c.js
|+-d_test.js
+-e_test.js
+-test.js
Thefollowingpatternswouldmatchthefollowingfiles:
test.js:
test.js
test/*.js:
/test/b.js
/test/c.js
/test/d_test.js
Writingtests
233
test/**/*.js:
/test/a/a1.js
/test/a/a2.js
/test/a/test.js
/test/b.js
/test/c.js
/test/d_test.js
**/test.js:
/test/a/test.js
**/*test.js:
/test/a/test.js
/test/d_test.js
/e_test.js
/test.js
Evenifmultiplepatternsmatchthesamefilethetestsinthatfilewillonlyberunonce.
Theorderoftestsisalwaysdeterminedbythefilepaths,nottheorderinwhichtheyarematchedorspecifiedinthemanifest.
TeststructureMochatestsuitescanbedefinedusingoneofthreeinterfaces:BDD,TDDorExports.
TheQUnitinterfaceofMochaisnotsupportedinArangoDB.
LikeallArangoDBcode,testcodeisalwayssynchronous.
BDDinterface
TheBDDinterfacedefinestestsuitesusingthedescribefunctionandeachtestcaseisdefinedusingtheitfunction:
'usestrict';
constassert=require('assert');
consttrueThing=true;
describe('Truethings',()=>{
it('aretrue',()=>{
assert.equal(trueThing,true);
});
});
TheBDDinterfacealsooffersthealiascontextfordescribeandspecifyforit.
Testfixturescanbehandledusingbeforeandafterforsuite-widefixturesandbeforeEachandafterEachforper-testfixtures:
describe('Falsethings',()=>{
letfalseThing;
before(()=>{
falseThing=!true;
});
it('arefalse',()=>{
assert.equal(falseThing,false);
});
});
TDDinterface
TheTDDinterfacedefinestestsuitesusingthesuitefunctionandeachtestcaseisdefinedusingthetestfunction:
'usestrict';
constassert=require('assert');
consttrueThing=true;
suite('Truethings',()=>{
test('aretrue',()=>{
Writingtests
234
assert.equal(trueThing,true);
});
});
TestfixturescanbehandledusingsuiteSetupandsuiteTeardownforsuite-widefixturesandsetupandteardownforper-testfixtures:
suite('Falsethings',()=>{
letfalseThing;
suiteSetup(()=>{
falseThing=!true;
});
test('arefalse',()=>{
assert.equal(falseThing,false);
});
});
Exportsinterface
TheExportsinterfacedefinestestcasesasmethodsofplainobjectpropertiesofthemodule.exportsobject:
'usestrict';
constassert=require('assert');
consttrueThing=true;
exports['Truethings']={
'aretrue':function(){
assert.equal(trueThing,true);
}
};
Thekeysbefore,after,beforeEachandafterEacharespecial-casedandbehavelikethecorrespondingfunctionsintheBDDinterface:
letfalseThing;
exports['Falsethings']={
before(){
falseThing=false;
},
'arefalse':function(){
assert.equal(falseThing,false);
}
};
Assertions
ArangoDBprovidestwobundledmodulestodefineassertions:
assertcorrespondstotheNode.jsassertmodule,providinglow-levelassertionsthatcanoptionallyspecifyanerrormessage.
chaiisthepopularChaiAssertionLibrary,providingbothBDDandTDDstyleassertionsusingafamiliarsyntax.
Writingtests
235
Cross-OriginResourceSharing(CORS)TouseCORSinyourFoxxservicesyoufirstneedtoconfigureArangoDBforCORS.Asof3.2Foxxwillthenautomaticallywhitelistallresponseheadersastheyareused.
IfyouwantmorecontroloverthewhitelistorareusinganolderversionofArangoDByoucansetthefollowingresponseheadersinyourrequesthandler:
access-control-expose-headers:acomma-separatedlistofresponseheaders.Thisdefaultstoalistofallheaderstheresponseisactuallyusing(butnotincludinganyaccess-controlheaders).
access-control-allow-credentials:canbesetto"false"toforbidexposingcookies.ThedefaultvaluedependsonwhetherArangoDBtruststheorigin.Seethenotesonhttp.trusted-origin.
NotethatitisnotpossibletooverridetheseheadersfortheCORSpreflightresponse.Itisthereforenotpossibletoacceptcredentialsorcookiesonlyforindividualroutes,servicesordatabases.TheoriginneedstobetrustedaccordingtothegeneralArangoDBconfiguration(seeabove).
CrossOrigin
236
FoxxscriptsandqueuedjobsFoxxletsyoudefinescriptsthatcanbeexecutedaspartoftheinstallationandremovalprocess,invokedmanuallyorscheduledtorunatalatertimeusingthejobqueue.
Toregisteryourscript,justaddascriptssectiontoyourservicemanifest:
{
...
"scripts":{
"setup":"scripts/setup.js",
"send-mail":"scripts/send-mail.js"
}
...
}
Thescriptsyoudefineinyourservicemanifestcanbeinvokedfromthewebinterfaceintheservice'ssettingspagewiththeScriptsdropdown.
Youcanalsousethescriptsasqueuedjobs:
'usestrict';
constqueues=require('@arangodb/foxx/queues');
queues.get('default').push(
{mount:'/my-service-mount-point',name:'send-mail'},
{to:'[email protected]',body:'Hello'}
);
ScriptargumentsandreturnvaluesIfthescriptwasinvokedwithanyarguments,youcanaccessthemusingthemodule.context.argvarray.
Toreturndatafromyourscript,youcanassignthedatatomodule.exportsasusual.PleasenotethatthisdatawillbeconvertedtoJSON.
Anyerrorsraisedbythescriptwillbehandleddependingonhowthescriptwasinvoked:
ifthescriptwasinvokedfromtheHTTPAPI(e.g.usingthewebinterface),itwillreturnanerrorresponseusingtheexception'sstatusCodepropertyifspecifiedor500.ifthescriptwasinvokedfromaFoxxjobqueue,thejob'sfailurecounterwillbeincrementedandthejobwillberescheduledormarkedasfailedifnoattemptsremain.
Examples
Let'ssayyouwanttodefineascriptthattakestwonumericvaluesandreturnstheresultofmultiplyingthem:
'usestrict';
constassert=require('assert');
constargv=module.context.argv;
assert.equal(argv.length,2,'Expectedexactlytwoarguments');
assert.equal(typeofargv[0],'number','Expectedfirstargumenttobeanumber');
assert.equal(typeofargv[1],'number','Expectedsecondargumenttobeanumber');
module.exports=argv[0]*argv[1];
LifecycleScriptsFoxxrecognizeslifecyclescriptsiftheyaredefinedandwillinvokethemduringtheinstallation,updateandremovalprocessoftheserviceifyouwant.
Scriptsandqueuedjobs
237
Thefollowingscriptsarecurrentlyrecognizedaslifecyclescriptsbytheirname:"setup"and"teardown".
SetupScript
ThesetupscriptwillbeexecutedwithoutargumentsduringtheinstallationofyourFoxxservice.
Thesetupscriptmaybeexecutedmorethanonceandshouldthereforebetreatedasreentrant.Runningthesamesetupscriptagainshouldnotresultinanyerrorsorduplicatedata.
Thesetupscriptistypicallyusedtocreatecollectionsyourserviceneedsorinsertseeddatalikeinitialadministrativeuseraccountsandsoon.
Examples
'usestrict';
constdb=require('@arangodb').db;
consttextsCollectionName=module.context.collectionName('texts');
//`textsCollectionName`isnowtheprefixednameofthisservice's"texts"collection.
//e.g."example_texts"iftheservicehasbeenmountedat`/example`
if(db._collection(textsCollectionName)===null){
constcollection=db._create(textsCollectionName);
collection.save({text:'entry1fromcollectiontexts'});
collection.save({text:'entry2fromcollectiontexts'});
collection.save({text:'entry3fromcollectiontexts'});
}else{
console.debug(`collection${texts}alreadyexists.Leavingituntouched.`);
}
TeardownScript
TheteardownscriptwillbeexecutedwithoutargumentsduringtheremovalofyourFoxxservice.
Itcanalsooptionallybeexecutedbeforeupgradinganservice.
Thisscripttypicallyremovesthecollectionsand/ordocumentscreatedbyyourservice'ssetupscript.
Examples
'usestrict';
constdb=require('@arangodb').db;
consttextsCollection=module.context.collection('texts');
if(textsCollection){
textsCollection.drop();
}
Queuesconstqueues=require('@arangodb/foxx/queues')
Foxxallowsdefiningjobqueuesthatletyouperformsloworexpensiveactionsasynchronously.Thesequeuescanbeusedtosende-mails,callexternalAPIsorperformotheractionsthatyoudonotwanttoperformdirectlyorwanttoretryonfailure.
enableordisabletheFoxxqueuesfeature--foxx.queuesflagIftrue,theFoxxqueueswillbeavailableandjobsinthequeueswillbeexecutedasynchronously.Thedefaultistrue.Whensettofalsethequeuemanagerwillbedisabledandanyjobsarepreventedfrombeingprocessed,whichmayreduceCPUloadabit.PleasenotethatFoxxjobqueuesaredatabase-specific.Queuesandjobsarealwaysrelativetothedatabaseinwhichtheyarecreatedoraccessed.
pollintervalforFoxxqueues--foxx.queues-poll-intervalvalueThepollintervalfortheFoxxqueuesmanager.Thevalueisspecifiedinseconds.LowervalueswillmeanmoreimmediateandmorefrequentFoxxqueuejobexecution,butwillmakethequeuethreadwakeupandquerythequeuesmoreoften.Whensettoalowvalue,thequeuethreadmightcauseCPUload.Thedefaultis1second.IfFoxx
Scriptsandqueuedjobs
238
queuesarenotusedmuch,thenthisvaluemaybeincreasedtomakethequeuesthreadwakeupless.Forthelow-levelfunctionalityseethechapteronthetaskmanagementmodule.
Creatingorupdatingaqueue
queues.create(name,[maxWorkers]):Queue
Returnsthequeueforthegivenname.Ifthequeuedoesnotexist,anewqueuewiththegivennamewillbecreated.IfaqueuewiththegivennamealreadyexistsandmaxWorkersisset,thequeue'smaximumnumberofworkerswillbeupdated.Thequeuewillbecreatedinthecurrentdatabase.
Arguments
name:string
Nameofthequeuetocreate.
maxWorkers:number(Default:1)
Themaximumnumberofworkers.
Examples
//Createaqueuewiththedefaultnumberofworkers(i.e.one)
constqueue1=queues.create("my-queue");
//Createaqueuewithagivennumberofworkers
constqueue2=queues.create("another-queue",2);
//Updatethenumberofworkersofanexistingqueue
constqueue3=queues.create("my-queue",10);
//queue1andqueue3refertothesamequeue
assertEqual(queue1,queue3);
Fetchinganexistingqueuequeues.get(name):Queue
Returnsthequeueforthegivenname.Ifthequeuedoesnotexistanexceptionisthrowninstead.
Thequeuewillbelookedupinthecurrentdatabase.
Arguments
name:string
Nameofthequeuetofetch.
Examples
Ifthequeuedoesnotyetexistanexceptionisthrown:
queues.get("some-queue");
//Error:Queuedoesnotexist:some-queue
//at...
Otherwisethequeuewillbereturned:
constqueue1=queues.create("some-queue");
constqueue2=queues.get("some-queue");
assertEqual(queue1,queue2);
Deletingaqueue
queues.delete(name):boolean
Returnstrueifthequeuewasdeletedsuccessfully.Ifthequeuedidnotexist,itreturnsfalseinstead.Thequeuewillbelookedupanddeletedinthecurrentdatabase.
Scriptsandqueuedjobs
239
Whenaqueueisdeleted,jobsonthatqueuewillnolongerbeexecuted.
Deletingaqueuewillnotdeleteanyjobsonthatqueue.
Arguments
name:string
Nameofthequeuetodelete.
Examples
constqueue=queues.create("my-queue");
queues.delete("my-queue");//true
queues.delete("my-queue");//false
Addingajobtoaqueue
queue.push(script,data,[opts]):string
Thejobwillbeaddedtothespecifiedqueueinthecurrentdatabase.
Returnsthejobid.
Arguments
script:object
Ajobtypedefinition,consistingofanobjectwiththefollowingproperties:
name:string
Nameofthescriptthatwillbeinvoked.
mount:string
Mountpathoftheservicethatdefinesthescript.
backOff:Function|number(Default:1000)
Eitherafunctionthattakesthenumberoftimesthejobhasfailedbeforeasinputandreturnsthenumberofmillisecondstowaitbeforetryingthejobagain,orthedelaytobeusedtocalculateanexponentialback-off,or0fornodelay.
maxFailures:number|Infinity(Default:0):
Numberoftimesasinglerunofajobwillbere-triedbeforeitismarkedas"failed".AnegativevalueorInfinitymeansthatthejobwillbere-triedonfailureindefinitely.
schema:Schema(optional)
Schematovalidateajob'sdataagainstbeforeenqueuingthejob.
preprocess:Function(optional)
Functiontopre-processajob's(validated)databeforeserializingitinthequeue.
data:any
Jobdataofthejob;mustbeserializabletoJSON.
opts:object(optional)
Objectwithanyofthefollowingproperties:
success:Function(optional)
Functiontobecalledafterthejobhasbeencompletedsuccessfully.
failure:Function(optional)
Functiontobecalledafterthejobhasfailedtoomanytimes.
Scriptsandqueuedjobs
240
delayUntil:number|Date(Default:Date.now())
Timestampinmilliseconds(orDateinstance)untilwhichtheexecutionofthejobshouldbedelayed.
backOff:Function|number(Default:1000)
Seescript.backOff.
maxFailures:number|Infinity(Default:0):
Seescript.maxFailures.
repeatTimes:number|Function(Default:0)
Ifsettoapositivenumber,thejobwillberepeatedthismanytimes(notcountingrecoverywhenusingmaxFailures).IfsettoanegativenumberorInfinity,thejobwillberepeatedindefinitely.Ifsetto0thejobwillnotberepeated.
repeatUntil:number|Date(optional)
Ifthejobissettoautomaticallyrepeat,thiscanbesettoatimestampinmilliseconds(orDateinstance)afterwhichthejobwillnolongerrepeat.Settingthisvaluetozero,anegativevalueorInfinityhasnoeffect.
repeatDelay:number(Default:0)
Ifthejobissettoautomaticallyrepeat,thiscanbesettoanon-negativevaluetosetthenumberofmillisecondsforwhichthejobwillbedelayedbeforeitisstartedagain.
NotethatifyoupassafunctionforthebackOffcalculation,successcallbackorfailurecallbackoptionsthefunctionwillbeserializedtothedatabaseasastringandthereforemustnotrelyonanyexternalscopeorexternalvariables.
Whenthejobissettoautomaticallyrepeat,thefailurecallbackwillonlybeexecutedwhenarunofthejobhasfailedmorethanmaxFailurestimes.NotethatifthejobfailsandmaxFailuresisset,itwillberescheduledaccordingtothebackOffuntilithaseitherfailedtoomanytimesorcompletedsuccessfullybeforebeingscheduledaccordingtotherepeatDelayagain.RecoveryattemptsbymaxFailuresdonotcounttowardsrepeatTimes.
Thesuccessandfailurecallbacksreceivethefollowingarguments:
result:any
Thereturnvalueofthescriptforthecurrentrunofthejob.
jobData:any
Thedatapassedtothismethod.
job:object
ArangoDBdocumentrepresentingthejob'scurrentstate.
Examples
Let'ssaywehaveanservicemountedat/mailerthatprovidesascriptcalledsend-mail:
'usestrict';
constqueues=require('@arangodb/foxx/queues');
constqueue=queues.create('my-queue');
queue.push(
{mount:'/mailer',name:'send-mail'},
{to:'[email protected]',body:'Helloworld'}
);
Thiswillnotwork,becauselogwasdefinedoutsidethecallbackfunction(thecallbackmustbeserializabletoastring):
//WARNING:THISDOESNOTWORK!
'usestrict';
constqueues=require('@arangodb/foxx/queues');
constqueue=queues.create('my-queue');
constlog=require('console').log;//outsidethecallback'sfunctionscope
queue.push(
{mount:'/mailer',name:'send-mail'},
Scriptsandqueuedjobs
241
{to:'[email protected]',body:'Helloworld'},
{success:function(){
log('Yay!');//throws'logisnotdefined'
}}
);
Here'sanexampleofajobthatwillbeexecutedevery5secondsuntiltomorrow:
'usestrict';
constqueues=require('@arangodb/foxx').queues;
constqueue=queues.create('my-queue');
queue.push(
{mount:'/mailer',name:'send-mail'},
{to:'[email protected]',body:'Helloworld'},
{
repeatTimes:Infinity,
repeatUntil:Date.now()+(24*60*60*1000),
repeatDelay:5*1000
}
);
Fetchingajobfromthequeue
queue.get(jobId):Job
Createsaproxyobjectrepresentingajobwiththegivenjobid.
Thejobwillbelookedupinthespecifiedqueueinthecurrentdatabase.
ReturnsthejobforthegivenjobId.Propertiesofthejobobjectwillbefetchedwhenevertheyarereferencedandcannotbemodified.
Arguments
jobId:string
Theidofthejobtocreateaproxyobjectfor.
Examples
constjobId=queue.push({mount:'/logger',name:'log'},'HelloWorld!');
constjob=queue.get(jobId);
assertEqual(job.id,jobId);
Deletingajobfromthequeue
queue.delete(jobId):boolean
Deletesajobwiththegivenjobid.Thejobwillbelookedupanddeletedinthespecifiedqueueinthecurrentdatabase.
Arguments
jobId:string
Theidofthejobtodelete.
Returnstrueifthejobwasdeletedsuccessfully.Ifthejobdidnotexistitreturnsfalseinstead.
Fetchinganarrayofjobsinaqueue
Examples
constlogScript={mount:'/logger',name:'log'};
queue.push(logScript,'HelloWorld!',{delayUntil:Date.now()+50});
assertEqual(queue.pending(logScript).length,1);
//50mslater...
assertEqual(queue.pending(logScript).length,0);
assertEqual(queue.progress(logScript).length,1);
//evenlater...
assertEqual(queue.progress(logScript).length,0);
Scriptsandqueuedjobs
242
assertEqual(queue.complete(logScript).length,1);
Fetchinganarrayofpendingjobsinaqueuequeue.pending([script]):Array<string>
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus"pending",optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
script:object(optional)
Anobjectwiththefollowingproperties:
name:stringNameofthescript.
mount:stringMountpathoftheservicedefiningthescript.
Fetchinganarrayofjobsthatarecurrentlyinprogress
queue.progress([script])
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus"progress",optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
script:object(optional)
Anobjectwiththefollowingproperties:
name:stringNameofthescript.
mount:stringMountpathoftheservicedefiningthescript.
Fetchinganarrayofcompletedjobsinaqueuequeue.complete([script]):Array<string>
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus"complete",optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
script:object(optional)
Anobjectwiththefollowingproperties:
name:stringNameofthescript.
mount:stringMountpathoftheservicedefiningthescript.
Fetchinganarrayoffailedjobsinaqueue
queue.failed([script]):Array<string>
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus"failed",optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
Scriptsandqueuedjobs
243
script:object(optional)
Anobjectwiththefollowingproperties:
name:stringNameofthescript.
mount:stringMountpathoftheservicedefiningthescript.
Fetchinganarrayofalljobsinaqueue
queue.all([script]):Array<string>
Returnsanarrayofjobidsofalljobsinthegivenqueue,optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
script:object(optional)
Anobjectwiththefollowingproperties:
name:stringNameofthescript.
mount:stringMountpathoftheservicedefiningthescript.
Abortingajobjob.abort():void
Abortsanon-completedjob.
Setsajob'sstatusto"failed"ifitisnotalready"complete",withoutcallingthejob'sonFailurecallback.
Scriptsandqueuedjobs
244
Migrating2.xservicesto3.0WhenmigratingservicesfromolderversionsofArangoDBitisgenerallyrecommendedyoumakesuretheyworkinlegacycompatibilitymode,whichcanalsoserveasastop-gapsolution.
ThischapteroutlinesthemajordifferencesintheFoxxAPIbetweenArangoDB2.8andArangoDB3.0.
Generalchanges
TheconsoleobjectinlaterversionsofArangoDB2.ximplementedaspecialFoxxconsoleAPIandwouldoptionallylogmessagestoacollection.ArangoDB3.0restorestheoriginalbehaviourwhereconsoleisthesameobjectavailablefromtheconsolemodule.
Migrating2.xservices
245
Migratingfrompre-2.8WhenmigratingfromaversionolderthanArangoDB2.8pleasenotethatstartingwithArangoDB2.8thebehaviouroftherequirefunctionmorecloselymimicsthebehaviourobservedinNode.jsandmodulebundlersforbrowsers,e.g.:
Inafile/routes/examples.js(relativetotherootfolderoftheservice):
require('./my-module')willbeattemptedtoberesolvedinthefollowingorder:
1. /routes/my-module(relativetoserviceroot)2. /routes/my-module.js(relativetoserviceroot)3. /routes/my-module.json(relativetoserviceroot)4. /routes/my-module/index.js(relativetoserviceroot)5. /routes/my-module/index.json(relativetoserviceroot)
require('lodash')willbeattemptedtoberesolvedinthefollowingorder:
1. /routes/node_modules/lodash(relativetoserviceroot)2. /node_modules/lodash(relativetoserviceroot)3. ArangoDBmodulelodash4. Nodecompatibilitymodulelodash5. BundledNPMmodulelodash
require('/abs/path')willbeattemptedtoberesolvedinthefollowingorder:
1. /abs/path(relativetofilesystemroot)2. /abs/path.js(relativetofilesystemroot)3. /abs/path.json(relativetofilesystemroot)4. /abs/path/index.js(relativetofilesystemroot)5. /abs/path/index.json(relativetofilesystemroot)
ThisbehaviourisincompatiblewiththesourcecodegeneratedbytheFoxxgeneratorinthewebinterfacebeforeArangoDB2.8.
Note:Theorg/arangodbmoduleisaliasedtothenewname@arangodbinArangoDB3.0.0andthe@arangodbmodulewasaliasedtotheoldnameorg/arangodbinArangoDB2.8.0.Eitheronewillworkin2.8and3.0butoutsideoflegacyservicesyoushoulduse@arangodbgoingforward.
FoxxqueueInArangoDB2.6FoxxintroducedanewwaytodefinequeuedjobsusingFoxxscriptstoreplacethefunction-basedjobtypedefinitionswhichwerecausingproblemswhenrestartingtheserver.Thefunction-basedjobshavebeenremovedin2.7andarenolongersupportedatall.
CoffeeScriptArangoDB3.0nolongerprovidesbuilt-insupportforCoffeeScriptsourcefiles,eveninlegacycompatibilitymode.IfyouwanttouseanalternativelanguagelikeCoffeeScript,makesuretopre-compiletherawsourcefilestoJavaScriptandusethecompiledJavaScriptfilesintheservice.
TherequestmoduleThe@arangodb/requestmodulewhenusedwiththejsonoptionpreviouslyoverwrotethestringinthebodypropertyoftheresponseobjectoftheresponsewiththeparsedJSONbody.In2.8thiswaschangedsotheparsedJSONbodyisaddedasthejsonpropertyoftheresponseobjectinadditiontooverwritingthebodyproperty.In3.0andlater(includinglegacycompatibilitymode)thebodypropertyisnolongeroverwrittenandmustusethejsonpropertyinstead.Notethatthisonlyaffectscodeusingthejsonoptionwhenmakingtherequest.
Migratingfrompre-2.8
246
BundledNPMmodules
ThebundledNPMmoduleshavebeenupgradedandmayincludebackwards-incompatiblechanges,especiallytheAPIofjoihaschangedseveraltimes.Ifindoubtyoushouldbundleyourownversionsofthesemodulestoensurespecificversionswillbeused.
Theutilitymodulelodashisnowavailableandshouldbeusedinsteadofunderscore,butbothmoduleswillcontinuetobeprovided.
Migratingfrompre-2.8
247
ManifestManyofthefieldsthatwererequiredinArangoDB2.xarenowoptionalandcanbesafelyomitted.
ToavoidcompatibilityproblemswithfutureversionsofArangoDByoushouldalwaysspecifytheenginesfield,e.g.:
{
"engines":{
"arangodb":"^3.0.0"
}
}
Controllers&exports
PreviouslyFoxxdistinguishedbetweenexportsandcontrollers,eachofwhichcouldbespecifiedasanobject.InArangoDB3.0thesehavebeenmergedintoasinglemainfieldspecifyinganentryfile.
Theeasiestwaytomigrateservicesusingmultipleexportsand/orcontrollersistocreateaseparateentryfilethatimportsthesefiles:
Old(manifest.json):
{
"exports":{
"doodads":"doodads.js",
"dingbats":"dingbats.js"
},
"controllers":{
"/doodads":"routes/doodads.js",
"/dingbats":"routes/dingbats.js",
"/":"routes/root.js"
}
}
New(manifest.json):
{
"main":"index.js"
}
New(index.js):
'usestrict';
module.context.use('/doodads',require('./routes/doodads'));
module.context.use('/dingbats',require('./routes/dingbats'));
module.context.use('/',require('./routes/root'));
module.exports={
doodads:require('./doodads'),
dingbats:require('./dingbats')
};
IndexredirectIfyoupreviouslydidnotdefinethedefaultDocumentfield,pleasenotethatinArangoDB3.0thefieldwillnolongerdefaulttothevalueindex.htmlwhenomitted:
Old:
{
//nodefaultDocument
}
manifest.json
248
New:
{
"defaultDocument":"index.html"
}
ThisalsomeansitisnolongernecessarytospecifythedefaultDocumentfieldwithanemptyvaluetopreventtheredirectandbeabletoserverequestsatthe/(root)pathofthemountpoint:
Old:
{
"defaultDocument":""
}
New:
{
//nodefaultDocument
}
AssetsTheassetsfieldisnolongersupportedinArangoDB3.0outsideoflegacycompatibilitymode.
Ifyoupreviouslyusedthefieldtoserveindividualfilesas-isyoucansimplyusethefilesfieldinstead:
Old:
{
"assets":{
"client.js":{
"files":["assets/client.js"],
"contentType":"application/javascript"
}
}
}
New:
{
"files":{
"client.js":{
"path":"assets/client.js",
"type":"application/javascript"
}
}
}
Ifyoureliedonbeingabletospecifymultiplefilesthatshouldbeconcatenated,youwillhavetousebuildtoolsoutsideofArangoDBtopreparethesefilesaccordingly.
RootelementTherootElementfieldisnolongersupportedandhasbeenremovedentirely.
IfyourcontrollersreliedonthisfieldbeingavailableyouneedtoadjustyourschemasandroutestobeabletohandlethefullJSONstructureofincomingdocuments.
Systemservices
manifest.json
249
TheisSystemfieldisnolongersupported.ThepresenceorabsenceofthefieldhadnoeffectinmostrecentversionsofArangoDB2.xandhasnowbeenremovedentirely.
manifest.json
250
TheapplicationcontextTheglobalapplicationContextvariableavailableinFoxxmoduleshasbeenreplacedwiththecontextattributeofthemodulevariable.Forconsistencyitisnowreferredtoastheservicecontextthroughoutthisdocumentation.
SomemethodsoftheservicecontexthavechangedinArangoDB3.0:
fileName()nowbehaveslikepath()didinArangoDB2.xpath()hasbeenremoved(usefileName()instead)foxxFileName()hasbeenremoved(usefileName()instead)
Additionallytheversionandnameattributeshavebeenremovedandcannowonlybeaccessedviathemanifestattribute(asmanifest.versionandmanifest.name).Notethatthecorrespondingmanifestfieldsarenowoptionalandmaybeomitted.
Theoptionsattributehasalsobeenremovedasitshouldbeconsideredanimplementationdetail.Youshouldinsteadaccessthedependenciesandconfigurationattributesdirectly.
Theinternal_prefixattribute(whichwasanaliasforbasePath)andtheinternalcommentandclearCommentsmethods(whichwereusedbythemagicaldocumentationcommentsinArangoDB2.x)havealsobeenremoved.
Theinternal_serviceattribute(whichprovidesaccesstotheserviceitself)hasbeenrenamedtoservice.
applicationContext
251
RepositoriesandmodelsPreviouslyFoxxwasheavilybuiltaroundtheconceptofrepositoriesandmodels,whichprovidedcomplexbutrarelynecessaryabstractionsontopofArangoDBcollectionsanddocuments.InArangoDB3.0thesehavebeenremovedentirely.
Repositoriesvscollections
RepositoriesmostlywrappedmethodsthatalreadyexistedonArangoDBcollectionobjectsandprimarilydealtwithconvertingbetweenplainArangoDBdocumentsandFoxxmodelinstances.InArangoDB3.0youcansimplyusethesecollectionsdirectlyandtreatdocumentsasplainJavaScriptobjects.
Old:
'usestrict';
constFoxx=require('org/arangodb/foxx');
constmyRepo=newFoxx.Repository(
applicationContext.collection('myCollection'),
{model:Foxx.Model}
);
//...
constmodels=myRepo.byExample({color:'green'});
res.json(models.map(function(model){
returnmodel.forClient();
}));
New:
'usestrict';
constmyDocs=module.context.collection('myCollection');
//...
constdocs=myDocs.byExample({color:'green'});
res.json(docs);
Schemavalidation
ThemainpurposeofmodelsinArangoDB2.xwastovalidateincomingdatausingjoischemas.InmorerecentversionsofArangoDB2.xitwasalreadypossibletopasstheseschemasdirectlyinmostplaceswhereamodelwasexpectedasanargument.Theonlydifferenceisthatschemasshouldnowbeconsideredthedefault.
IfyoupreviouslyreliedontheautomaticvalidationofFoxxmodelinstanceswhensettingattributesorinstantiatingmodelsfromuntrusteddata,youcansimplyusetheschema'svalidatemethoddirectly.
Old:
'usestrict';
constjoi=require('joi');
constmySchema={
name:joi.string().required(),
size:joi.number().required()
};
constFoxx=require('org/arangodb/foxx');
constMyModel=Foxx.Model.extend({schema:mySchema});
//...
constmodel=newMyModel(req.json());
if(!model.isValid){
res.status(400);
RepositoriesandModels
252
res.write('Badrequest');
return;
}
New:
'usestrict';
constjoi=require('joi');
//Notethisisnowwrappedinajoi.object()
constmySchema=joi.object({
name:joi.string().required(),
size:joi.number().required()
}).required();
//...
constresult=mySchema.validate(req.body);
if(result.errors){
res.status(400);
res.write('Badrequest');
return;
}
Migratingmodels
Whilemostusecasesformodelscannowbereplacedwithplainjoischemas,thereisstilltheconceptofa"model"inFoxxinArangoDB3.0althoughitisquitedifferentfromFoxxmodelsinArangoDB2.x.
AmodelinFoxxnowreferstoaplainJavaScriptobjectwithanoptionalschemaattributeandtheoptionalmethodsforClientandfromClient.Modelscanbeusedinsteadofplainjoischemastodefinerequestandresponsebodiesbuttherearenomodel"instances"inArangoDB3.0.
Old:
'usestrict';
const_=require('underscore');
constjoi=require('joi');
constFoxx=require('org/arangodb/foxx');
constMyModel=Foxx.Model.extend({
schema:{
name:joi.string().required(),
size:joi.number().required()
},
forClient(){
return_.omit(this.attributes,['_key','_id','_rev']);
}
});
//...
ctrl.get(/*...*/)
.bodyParam('body',{type:MyModel});
New:
'usestrict';
const_=require('lodash');
constjoi=require('joi');
constMyModel={
schema:joi.object({
name:joi.string().required(),
size:joi.number().required()
}).required(),
forClient(data){
return_.omit(data,['_key','_id','_rev']);
}
};
RepositoriesandModels
253
//...
router.get(/*...*/)
.body(MyModel);
TriggersWhensaving,updating,replacingordeletingmodelsinArangoDB2.xusingtherepositorymethodstherepositoryandmodelwouldfireeventsthatcouldbesubscribedtoinordertoperformside-effects.
Notethatevenin2.xtheseeventswouldnotfirewhenusingqueriesormanipulatingdocumentsinanyotherwaythanusingthespecificrepositorymethodsthatoperatedonindividualdocuments.
ThisbehaviourisnolongeravailableinArangoDB3.0butcanbeemulatedbyusinganEventEmitterdirectlyifitisnotpossibletosolvetheproblemdifferently:
Old:
'usestrict';
constFoxx=require('org/arangodb/foxx');
constMyModel=Foxx.Model.extend({
//...
},{
afterRemove(){
console.log(this.get('name'),'wasremoved');
}
});
//...
constmodel=myRepo.firstExample({name:'myName'});
myRepo.remove(model);
//->"myNamewasremovedsuccessfully"
New:
'usestrict';
constEventEmitter=require('events');
constemitter=newEventEmitter();
emitter.on('afterRemove',function(doc){
console.log(doc.name,'wasremoved');
});
//...
constdoc=myDocs.firstExample({name:'myName'});
myDocs.remove(doc);
emitter.emit('afterRemove',doc);
//->"myNamewasremovedsuccessfully"
Orsimply:
'usestrict';
functionafterRemove(doc){
console.log(doc.name,'wasremoved');
}
//...
constdoc=myDocs.firstExample({name:'myName'});
myDocs.remove(doc);
afterRemove(doc);
//->"myNamewasremovedsuccessfully"
RepositoriesandModels
254
ControllersvsroutersFoxxControllershavebeenreplacedwithrouters.Thisismorethanacosmeticchangeastherearesignificantdifferencesinbehaviour:
Controllerswereautomaticallymountedwhenthefiledefiningthemwasexecuted.Routersneedtobeexplicitlymountedusingthemodule.context.usemethod.Routerscanalsobeexported,importedandevennested.Thismakesiteasiertosplitupcomplexroutingtreesacrossmultiplefiles.
Old:
'usestrict';
constFoxx=require('org/arangodb/foxx');
constctrl=newFoxx.Controller(applicationContext);
ctrl.get('/hello',function(req,res){
//...
});
New:
'usestrict';
constcreateRouter=require('org/arangodb/foxx/router');
constrouter=createRouter();
//Ifyouareimportingthisfilefromyourentryfile("main"):
module.exports=router;
//Otherwise:module.context.use(router);
router.get('/hello',function(req,res){
//...
});
Somegeneralchangesinbehaviourthatmighttripyouup:
WhenspecifyingpathparameterswithschemasFoxxwillnowignoretherouteiftheschemadoesnotmatch(i.e./hello/foxxwillnolongermatch/hello/:numifnumspecifiesaschemathatdoesn'tmatchthevalue"foxx").Withcontrollersthiscouldpreviouslyresultinusersseeinga400(badrequest)errorwhentheyshouldinsteadbeserveda404(notfound)response.
WhenarequestismadewithanHTTPverbnotsupportedbyanendpoint,Foxxwillnowrespondwitha405(methodnotallowed)errorwithanappropriateAllowedheaderlistingthesupportedHTTPverbsforthatendpoint.
FoxxwillnolongerparseyourJSDoccommentstogenerateroutedocumentation(usethesummaryanddescriptionmethodsoftheendpointinstead).
TheapiDocumentationmethodnowlivesontheservicecontextandbehavesslightlydifferently.
ThereisnorouterequivalentfortheactivateAuthenticationandactivateSessionsmethods.Insteadyoushouldusethesessionmiddleware(seethesectiononsessionsbelow).
Thereisnodelaliasforthedeletemethodonrouters.IthasalwaysbeensafetousekeywordsasmethodnamesinFoxx,sotheuseofthisaliaswasalreadydiscouragedbefore.
TheallRoutesproxyisnolotavailableonroutersbutcaneasilybereplacedwithmiddlewareorchildrouters.
Controllers
255
TherequestcontextWhendefiningarouteonacontrollerthecontrollerwouldreturnanobjectcalledrequestcontext.Routersreturnasimilarobjectcalledendpoint.Routersalsoreturnendpointswhenmountingchildroutersormiddleware,asdoestheusemethodoftheservicecontext.
ThemaindifferencesbetweenthenewendpointsandtheobjectsreturnedbycontrollersinpreviousversionsofArangoDBare:
bodyParamisnowsimplycalledbody;itisnolongerneccessaryorpossibletogivethebodyanameandtherequestbodywillnotshowupintherequestparameters.It'salsopossibletospecifyaMIMEtype
body,queryParamandpathParamnowtakepositionargumentsinsteadofanobject.Forspecificsseetheendpointdocumentation.
notesisnowcalleddescriptionandtakesasinglestringargument.
onlyIfandonlyIfAuthenticatedarenolongeravailable;theycanbeemulatedwithmiddlewareifnecessary:
Old:
ctrl.get(/*...*/)
.onlyIf(function(req){
if(!req.user){
thrownewError('Notauthenticated!');
}
});
New:
router.use(function(req,res,next){
if(!req.arangoUser){
res.throw(403,'Notauthenticated!');
}
next();
});
router.get(/*...*/);
Controllers
256
ErrorhandlingTheerrorResponsemethodprovidedbycontrollerrequestcontextshasnoequivalentinrouterendpoints.Ifyouwanttohandlespecificerrortypeswithspecificstatuscodesyouneedtocatchthemexplicitly,eitherintherouteorinamiddleware:
Old:
ctrl.get('/puppies',function(req,res){
//Exceptionisthrownhere
})
.errorResponse(TooManyPuppiesError,400,'Somethingwentwrong!');
New:
ctrl.get('/puppies',function(req,res){
try{
//Exceptionisthrownhere
}catch(e){
if(!(einstanceofTooManyPuppiesError)){
throwe;
}
res.throw(400,'Somethingwentwrong!');
}
})
//The"error"methodmerelydocumentsthemeaning
//ofthestatuscodeandhasnoothereffect.
.error(400,'Throwniftherearetoomanypuppies.');
Notethaterrorscreatedwithhttp-errorsarestillhandledbyFoxxintelligently.Infactres.throwisjustahelpermethodforcreatingandthrowingtheseerrors.
Controllers
257
Before,afterandaroundThebefore,afterandaroundmethodscaneasilybereplacedbymiddleware:
Old:
letstart;
ctrl.before(function(req,res){
start=Date.now();
});
ctrl.after(function(req,res){
console.log('Requesthandledin',(Date.now()-start),'ms');
});
New:
router.use(function(req,res,next){
letstart=Date.now();
next();
console.log('Requesthandledin',(Date.now()-start),'ms');
});
Notethatunlikearoundmiddlewarereceivesthenextfunctionasthethirdargument(the"opts"argumenthasnoequivalent).
Controllers
258
RequestobjectsThenamesofsomeattributesoftherequestobjecthavebeenadjustedtomorecloselyalignwiththoseofthecorrespondingmethodsontheendpointobjectsandestablishedconventionsinotherJavaScriptframeworks:
req.urlParametersisnowcalledreq.pathParams
req.parametersisnowcalledreq.queryParams
req.params()isnowcalledreq.param()
req.requestTypeisnowcalledreq.method
req.compatibilityisnowcalledreq.arangoVersion
req.userisnowcalledreq.arangoUser
Someattributeshavebeenremovedorchanged:
req.cookieshasbeenremovedentirely(usereq.cookie(name))
req.requestBodyhasbeenremovedentirely(seebelow)
req.suffixisnowastringratherthananarray
Additionallythereq.serverandreq.clientattributesarenolongeravailable.Theinformationisnowexposedinawaythatcan(optionally)transparentlyhandleproxyforwardingheaders:
req.hostnamedefaultstoreq.server.address
req.portdefaultstoreq.server.port
req.remoteAddressdefaultstoclient.address
req.remotePortdefaultstoclient.port
Finally,thereq.cookiemethodnowtakesthesignedoptionsdirectly.
Old:
constsid=req.cookie('sid',{
signed:{
secret:'keyboardcat',
algorithm:'sha256'
}
});
New:
constsid=req.cookie('sid',{
secret:'keyboardcat',
algorithm:'sha256'
});
RequestbodiesThereq.bodyisnolongeramethodandnolongerautomaticallyparsesJSONrequestbodiesunlessarequestbodywasdefined.Thereq.rawBodynowcorrespondstothereq.rawBodyBufferofArangoDB2.xandisalsonolongeramethod.
Old:
ctrl.post('/',function(req,res){
constdata=req.body();
//...
});
Controllers
259
New:
router.post('/',function(req,res){
constdata=req.body;
//...
})
.body(['json']);
Orsimply:
constjoi=require('joi');
router.post('/',function(req,res){
constdata=req.body;
//...
})
.body(joi.object().optional());
Multipartrequests
Thereq.requestPartsmethodhasbeenremovedentirely.Ifyouneedtoacceptmultipartrequestbodies,youcansimplydefinetherequestbodyusingamultipartMIMEtypelikemultipart/form-data:
Old:
ctrl.post('/',function(req,res){
constparts=req.requestParts();
//...
});
New:
router.post('/',function(req,res){
constparts=req.body;
//...
})
.body(['multipart/form-data']);
Controllers
260
ResponseobjectsTheresponseobjecthasalotofnewmethodsinArangoDB3.0butotherwiseremainssimilartotheresponseobjectofpreviousversions:
Theres.sendmethodbehavesverydifferentlyfromhowthemethodwiththesamenamebehavedinArangoDB2.x:theconversionnowtakestheresponsebodydefinitionoftherouteintoaccount.Thereisanewmethodres.writethatimplementstheoldbehaviour.
Notethatconsecutivecallstores.writewillappendtotheresponsebodyratherthanreplacingitlikeres.send.
Theres.contentTypepropertyisalsonolongeravailable.IfyouwanttosettheMIMEtypeoftheresponsebodytoanexplicitvalueyoushouldsetthecontent-typeheaderinstead:
Old:
res.contentType='application/json';
res.body=JSON.stringify(results);
New:
res.set('content-type','application/json');
res.body=JSON.stringify(results);
Orsimply:
//setsthecontenttypetoJSON
//ifithasnotalreadybeenset
res.json(results);
Theres.cookiemethodnowtakesthesignedoptionsaspartoftheregularoptionsobject.
Old:
res.cookie('sid','abcdef',{
ttl:60*60,
signed:{
secret:'keyboardcat',
algorithm:'sha256'
}
});
New:
res.cookie('sid','abcdef',{
ttl:60*60,
secret:'keyboardcat',
algorithm:'sha256'
});
Controllers
261
DependencyinjectionThereisnoequivalentoftheaddInjectormethodavailableinArangoDB2.xcontrollers.Mostusecasescanbesolvedbysimplyusingplainvariablesbutifyouneedsomethingmoreflexibleyoucanalsousemiddleware:
Old:
ctrl.addInjector('magicNumber',function(){
returnMath.random();
});
ctrl.get('/',function(req,res,injected){
res.json(injected.magicNumber);
});
New:
functionmagicMiddleware(name){
return{
register(){
letmagic;
returnfunction(req,res,next){
if(!magic){
magic=Math.random();
}
req[name]=magic;
next();
};
}
};
}
router.use(magicMiddleware('magicNumber'));
router.get('/',function(req,res){
res.json(req.magicNumber);
});
Orsimply:
constmagicNumber=Math.random();
router.get('/',function(req,res){
res.json(magicNumber);
});
Controllers
262
SessionsThectrl.activateSessionsmethodandtherelatedutil-sessions-localFoxxservicehavebeenreplacedwiththeFoxxsessionsmiddleware.Itisnolongerpossibletousethebuilt-insessionstoragebutyoucansimplypassinanydocumentcollectiondirectly.
Old:
constlocalSessions=applicationContext.dependencies.localSessions;
constsessionStorage=localSessions.sessionStorage;
ctrl.activateSessions({
sessionStorage:sessionStorage,
cookie:{secret:'keyboardcat'}
});
ctrl.destroySession('/logout',function(req,res){
res.json({message:'Goodbye!'});
});
New:
constsessionMiddleware=require('@arangodb/foxx/sessions');
constcookieTransport=require('@arangodb/foxx/sessions/transports/cookie');
router.use(sessionMiddleware({
storage:module.context.collection('sessions'),
transport:cookieTransport('keyboardcat')
}));
router.post('/logout',function(req,res){
req.sessionStorage.clear(req.session);
res.json({message:'Goodbye!'});
});
Sessions
263
AuthandOAuth2Theutil-simple-authandutil-oauth2FoxxserviceshavebeenreplacedwiththeFoxxauthandFoxxOAuth2modules.Itisnolongernecessarytoinstalltheseservicesasdependenciesinordertousethefunctionality.
Old:
'usestrict';
constauth=applicationContext.dependencies.simpleAuth;
//...
constvalid=auth.verifyPassword(authData,password);
New:
'usestrict';
constcreateAuth=require('@arangodb/foxx/auth');
constauth=createAuth();//Usedefaultconfiguration
//...
constvalid=auth.verifyPassword(authData,password);
AuthandOAuth2
264
FoxxqueriesThecreateQuerymethodhasbeenremoved.ItcanbetriviallyreplacedwithplainJavaScriptfunctionsanddirectcallstothedb._querymethod:
Old:
'usestrict';
constFoxx=require('org/arangodb/foxx');
constquery=Foxx.createQuery({
query:'FORuIN_usersSORTu.userASCRETURNu[@propName]',
params:['propName'],
transform:function(results,uppercase){
return(
uppercase
?results[0].toUpperCase()
:results[0].toLowerCase()
);
}
});
query('user',true);
New:
'usestrict';
constdb=require('@arangodb').db;
constaql=require('@arangodb').aql;
functionquery(propName,uppercase){
constresults=db._query(aql`
FORuIN_users
SORTu.userASC
RETURNu[${propName}]
`);
return(
uppercase
?results[0].toUpperCase()
:results[0].toLowerCase()
);
}
query('user',true);
FoxxQueries
265
Legacycompatibilitymodefor2.8servicesArangoDB3continuestosupportFoxxserviceswrittenforArangoDB2.8byrunningtheminaspeciallegacycompatibilitymodethatprovidesaccesstosomeofthemodulesandAPIsnolongerprovidedin3.0andbeyond.
Note:LegacycompatibilitymodeisstrictlyintendedasatemporarystopgapsolutionforsupportingexistingserviceswhileupgradingtoArangoDB3.0andshouldnotbeconsideredapermanentfeatureofArangoDBorFoxx.
Inordertomarkanexistingserviceasalegacyservice,justmakesurethefollowingattributeisdefinedintheservicemanifest:
"engines":{
"arangodb":"^2.8.0"
}
ThissemanticversionrangedenotesthattheserviceisknowntoworkwithArangoDB2.8.0andsupportsallnewerversionsofArangoDBuptobutnotincluding3.0.0(noranydevelopmentversionof3.0.0andgreater).
Anysimilarversionrangethedoesnotinclude3.0.0orgreaterwillhavethesameeffect(e.g. 2.5.0willalsotriggerthelegacycompatibilitymode,aswill1.2.3,but>=2.8.0willnotasitindicatescompatibilitywithallversionsgreaterorequal2.8.0,notjustthosewithinthe2.xversionrange).
FeaturessupportedinlegacycompatibilitymodeLegacycompatibilitymodesupportstheoldmanifestformat,specifically:
mainisignoredcontrollerswillbemountedasin2.8exportswillbeexecutedasin2.8
AdditionallytheisSystemattributewillbeignoredifpresentbutdoesnotresultinawarninginlegacycompatibilitymode.
TheFoxxconsoleisavailableastheconsolepseudo-globalvariable(shadowingtheglobalconsoleobject).
TheservicecontextisavailableastheapplicationContextpseudo-globalvariableinthecontrollers,exports,scriptsandtestsasin2.8.Thefollowingadditionalpropertiesareavailableontheservicecontextinlegacycompatibilitymode:
path()isanaliasfor3.xfileName()(usingpath.jointobuildfilepaths)fileName()behavesasin2.x(usingfs.safeJointobuildfilepaths)foxxFileName()isanaliasfor2.xfileNameversionexposestheservicemanifest'sversionattributenameexposestheservicemanifest'snameattributeoptionsexposestheservice'srawoptions
Thefollowingmethodsareremovedontheservicecontextinlegacycompatibilitymode:
use()--use@arangodb/foxx/controllerinsteadapiDocumentation()--usecontroller.apiDocumentation()insteadregisterType()--notsupportedinlegacycompatibilitymode
Thefollowingmodulesthathavebeenremovedorreplacedin3.0.0areavailableinlegacycompatibilitymode:
@arangodb/foxx/authentication
@arangodb/foxx/console
@arangodb/foxx/controller
@arangodb/foxx/model
@arangodb/foxx/query
@arangodb/foxx/repository
@arangodb/foxx/schema
@arangodb/foxx/sessions
@arangodb/foxx/template_middleware
Legacycompatibilitymode
266
The@arangodb/foxxmodulealsoprovidesthesameexportsasin2.8,namely:
Controllerfrom@arangodb/foxx/controllercreateQueryfrom@arangodb/foxx/queryModelfrom@arangodb/foxx/modelRepositoryfrom@arangodb/foxx/repositorytoJSONSchemafrom@arangodb/foxx/schemagetExportsandrequireAppfrom@arangodb/foxx/managerqueuesfrom@arangodb/foxx/queues
Anyfeaturenotsupportedin2.8willalsonotworkinlegacycompatibilitymode.WhenmigratingfromanolderversionofArangoDBitisagoodideatomigratetoArangoDB2.8firstforaneasierupgradepath.
AdditionallypleasenotethedifferenceslaidoutinthechapterMigratingfrompre-2.8inthemigrationguide.
Legacycompatibilitymode
267
UsermanagementFoxxdoesnotprovideanyusermanagementoutoftheboxbutitisveryeasytorollyourownsolution:
Thesessionmiddlewareprovidesmechanismsforaddingsessionlogictoyourservice,usinge.g.acollectionorJSONWebTokenstostorethesessionsbetweenrequests.
Theauthmoduleprovidesutilitiesforbasicpasswordverificationandhashing.
Thefollowingexampleservicedemonstrateshowusermanagementcanbeimplementedusingthesebasicbuildingblocks.
Settingupthecollections
Let'ssaywewanttostoresessionsandusersincollections.Wecanusethesetupscripttomakesurethesecollectionsarecreatedbeforetheserviceismounted.
Firstaddasetupscripttoyourmanifestifitisn'talreadydefined:
"scripts":{
"setup":"scripts/setup.js"
}
Thencreatethesetupscriptwiththefollowingcontent:
'usestrict';
constdb=require('@arangodb').db;
constsessions=module.context.collectionName('sessions');
constusers=module.context.collectionName('users');
if(!db._collection(sessions)){
db._createDocumentCollection(sessions);
}
if(!db._collection(users)){
db._createDocumentCollection(users);
}
db._collection(users).ensureIndex({
type:'hash',
fields:['username'],
unique:true
});
Creatingtherouter
Thefollowingmainfiledemonstratesbasicusermanagement:
'usestrict';
constjoi=require('joi');
constcreateAuth=require('@arangodb/foxx/auth');
constcreateRouter=require('@arangodb/foxx/router');
constsessionsMiddleware=require('@arangodb/foxx/sessions');
constauth=createAuth();
constrouter=createRouter();
constusers=module.context.collection('users');
constsessions=sessionsMiddleware({
storage:module.context.collection('sessions'),
transport:'cookie'
});
module.context.use(sessions);
module.context.use(router);
router.get('/whoami',function(req,res){
Usermanagement
268
try{
constuser=users.document(req.session.uid);
res.send({username:user.username});
}catch(e){
res.send({username:null});
}
})
.description('Returnsthecurrentlyactiveusername.');
router.post('/login',function(req,res){
//Thismayreturnauserobjectornull
constuser=users.firstExample({
username:req.body.username
});
constvalid=auth.verify(
//Pretendtovalidateevenifnouserwasfound
user?user.authData:{},
req.body.password
);
if(!valid)res.throw('unauthorized');
//Logtheuserin
req.session.uid=user._key;
req.sessionStorage.save(req.session);
res.send({sucess:true});
})
.body(joi.object({
username:joi.string().required(),
password:joi.string().required()
}).required(),'Credentials')
.description('Logsaregistereduserin.');
router.post('/logout',function(req,res){
if(req.session.uid){
req.session.uid=null;
req.sessionStorage.save(req.session);
}
res.send({success:true});
})
.description('Logsthecurrentuserout.');
router.post('/signup',function(req,res){
constuser=req.body;
try{
//Createanauthenticationhash
user.authData=auth.create(user.password);
deleteuser.password;
constmeta=users.save(user);
Object.assign(user,meta);
}catch(e){
//Failedtosavetheuser
//We'llassumetheUniqueConstrainthasbeenviolated
res.throw('badrequest','Usernamealreadytaken',e);
}
//Logtheuserin
req.session.uid=user._key;
req.sessionStorage.save(req.session);
res.send({success:true});
})
.body(joi.object({
username:joi.string().required(),
password:joi.string().required()
}).required(),'Credentials')
.description('Createsanewuserandlogsthemin.');
Usermanagement
269
RelatedmodulesThesearesomeofthemodulesoutsideofFoxxyouwillfindusefulwhenwritingFoxxservices.
AdditionallytherearemodulesprovidingsomelevelofcompatibilitywithNode.jsaswellasanumberofbundledNPMmodules(likelodashandjoi).FormoreinformationonthesemodulesseetheJavaScriptmodulesappendix.
The @arangodbmodulerequire('@arangodb')
ThismoduleprovidesaccesstovariousArangoDBinternalsaswellasthreeofthemostimportantexportsnecessarytoworkwiththedatabaseinFoxx:db,aqlanderrors.
YoucanfindafulldescriptionofthismoduleintheArangoDBmoduleappendix.
The @arangodb/requestmodulerequire('@arangodb/request')
ThismoduleprovidesafunctionformakingHTTPrequeststoexternalservices.Notethatwhilethisallowscommunicatingwiththird-partyservicesitmayaffectdatabaseperformancebyblockingFoxxrequestsasArangoDBwaitsfortheremoteservicetorespond.Ifyouroutinelymakerequeststoslowexternalservicesandarenotdirectlyinterestedintheresponseitisprobablyabetterideatodelegatetheactualrequest/responsecycletoagatewayservicerunningoutsideArangoDB.
Youcanfindafulldescriptionofthismoduleintherequestmoduleappendix.
The @arangodb/general-graphmodulerequire('@arangodb/general-graph')
ThismoduleprovidesaccesstoArangoDBgraphdefinitionsandvariouslow-levelgraphoperationsinJavaScript.FormorecomplexqueriesitisprobablybettertouseAQLbutthismodulecanbeusefulinyoursetupandteardownscriptstocreateanddestroygraphdefinitions.
Formoreinformationseethechapteronthegeneralgraphmodule.
Relatedmodules
270
AuthenticationconstcreateAuth=require('@arangodb/foxx/auth');
Authenticatorsallowimplementingbasicpasswordmechanismusingsimplebuilt-inhashingfunctions.
ForafullexampleofsessionswithauthenticationandregistrationseetheexampleinthechapteronUserManagement.
CreatinganauthenticatorcreateAuth([options]):Authenticator
Createsanauthenticator.
Arguments
options:Object(optional)
Anobjectwiththefollowingproperties:
method:string(Default:"sha256")
Thehashingalgorithmtousetocreatepasswordhashes.Theauthenticatorwillbeabletoverifypasswordsagainsthashesusinganysupportedhashingalgorithm.Thisonlyaffectsnewhashescreatedbytheauthenticator.
Supportedvalues:
"md5"
"sha1"
"sha224"
"sha256"
"sha384"
"sha512"
saltLength:number(Default:16)
Lengthofthesaltsthatwillbegeneratedforpasswordhashes.
Returnsanauthenticator.
Creatingauthenticationdataobjectsauth.create(password):AuthData
Createsanauthenticationdataobjectforthegivenpasswordwiththefollowingproperties:
method:string
Themethodusedtogeneratethehash.
salt:string
Arandomsaltusedtogeneratethishash.
hash:string
Thehashstringitself.
Arguments
password:string
Apasswordtohash.
Returnstheauthenticationdataobject.
Authentication
271
Validatingpasswordsagainstauthenticationdataobjectsauth.verify([hash,[password]]):boolean
Verifiesthegivenpasswordagainstthegivenhashusingaconstanttimestringcomparison.
Arguments
hash:AuthData(optional)
Aauthenticationdataobjectgeneratedwiththecreatemethod.
password:string(optional)
Apasswordtoverifyagainstthehash.
Returnstrueifthehashmatchesthegivenpassword.Returnsfalseotherwise.
Authentication
272
OAuth1.0aconstcreateOAuth1Client=require('@arangodb/foxx/oauth1');
TheOAuth1moduleprovidesabstractionsoverOAuth1.0aproviderslikeTwitter,XINGandTumblr.
Examples
Thefollowingextendstheusermanagementexample:
constrouter=createRouter();
constoauth1=createOAuth1Client({
//We'lluseTwitterforthisexample
requestTokenEndpoint:'https://api.twitter.com/oauth/request_token',
authEndpoint:'https://api.twitter.com/oauth/authorize',
accessTokenEndpoint:'https://api.twitter.com/oauth/access_token',
activeUserEndpoint:'https://api.twitter.com/1.1/account/verify_credentials.json',
clientId:'keyboardcat',
clientSecret:'keyboardcat'
});
module.context.use('/oauth1',router);
//Seetheusermanagementexampleforsettingupthe
//sessionsandusersobjectsusedinthisexample
router.use(sessions);
router.post('/auth',function(req,res){
consturl=req.reverse('oauth1_callback');
constoauth_callback=req.makeAbsolute(url);
constrequestToken=oauth1.fetchRequestToken(oauth_callback);
if(requestToken.oauth_callback_confirmed!=='true'){
res.throw(500,'CouldnotfetchOAuthrequesttoken');
}
//Setrequesttokencookieforfiveminutes
res.cookie('oauth1_request_token',requestToken.oauth_token,{ttl:60*5});
//Redirecttotheprovider'sauthorizationURL
res.redirect(303,oauth1.getAuthUrl(requestToken.oauth_token));
});
router.get('/auth',function(req,res){
//MakesureCSRFcookiematchestheURL
constexpectedToken=req.cookie('oauth1_request_token');
if(!expectedToken||req.queryParams.oauth_token!==expectedToken){
res.throw(400,'CSRFmismatch.');
}
constauthData=oauth1.exchangeRequestToken(
req.queryParams.oauth_token,
req.queryParams.oauth_verifier
);
consttwitterToken=authData.oauth_token;
consttwitterSecret=authData.oauth_token_secret;
//Fetchtheactiveuser'sprofileinfo
constprofile=oauth1.fetchActiveUser(twitterToken,twitterSecret);
consttwitterId=profile.screen_name;
//TrytofindanexistinguserwiththeuserID
//(thisrequirestheuserscollection)
letuser=users.firstExample({twitterId});
if(user){
//UpdatethetwitterTokenifithaschanged
if(
user.twitterToken!==twitterToken||
user.twitterSecret!==twitterSecret
){
users.update(user,{twitterToken,twitterSecret});
}
}else{
//Createanewuserdocument
user={
username:`twitter:${twitterId}`,
twitterId,
twitterToken
OAuth1.0a
273
}
constmeta=users.save(user);
Object.assign(user,meta);
}
//Logtheuserin(thisrequiresthesessionmiddleware)
req.session.uid=user._key;
req.session.twitterToken=authData.twitterToken;
req.session.twitterSecret=authData.twitterSecret;
req.sessionStorage.save(req.session);
//Redirecttothedefaultroute
res.redirect(303,req.makeAbsolute('/'));
},'oauth1_callback')
.queryParam('oauth_token',joi.string().optional())
.queryParam('oauth_verifier',joi.string().optional());
CreatinganOAuth1.0aclientcreateOAuth1Client(options):OAuth1Client
CreatesanOAuth1.0aclient.
Arguments
options:Object
Anobjectwiththefollowingproperties:
requestTokenEndpoint:string
Thefully-qualifiedURLoftheprovider'sTemporaryCredentialsRequestendpoint.ThisURLisusedtofetchtheunauthenticatedtemporarycredentialsthatwillbeusedtogeneratetheauthorizationredirectfortheuser.
authEndpoint:string
Thefully-qualifiedURLoftheprovider'sResourceOwnerAuthorizationendpoint.ThisistheURLtheuserwillberedirectedtoinordertoauthorizetheOAuthconsumer(i.e.yourservice).
accessTokenEndpoint:string
Thefully-qualifiedURLoftheprovider'sTokenRequestendpoint.ThisURLisusedtoexchangetheauthenticatedtemporarycredentialsreceivedfromtheauthorizationredirectfortheactualtokencredentialsthatcanbeusedtomakerequeststotheAPIserver.
activeUserEndpoint:string(optional)
Thefully-qualifiedURLoftheprovider'sendpointforfetchingdetailsaboutthecurrentuser.
clientId:string
Theapplication'sClientID(orConsumerKey)fortheprovider.
clientSecret:string
Theapplication'sClientSecret(orConsumerSecret)fortheprovider.
signatureMethod:string(Default:"HMAC-SHA1")
ThecryptographicmethodthatwillbeusedtosignOAuth1.0arequests.Only"HMAC-SHA1-"and"PLAINTEXT"aresupportedatthistime.
Notethatmanyprovidersmaynotimplement"PLAINTEXT"asitexposestheClientSecretandoauth_token_secretinsteadofgeneratingasignature.
ReturnsanOAuth1.0aclientforthegivenprovider.
SettingupOAuth1.0aforTwitter
IfyouwanttouseTwitterastheOAuth1.0aprovider,usethefollowingoptions:
OAuth1.0a
274
requestTokenEndpoint:https://api.twitter.com/oauth/request_tokenauthEndpoint:https://api.twitter.com/oauth/authorizeaccessTokenEndpoint:https://api.twitter.com/oauth/access_tokenactiveUserEndpoint:https://api.twitter.com/1.1/account/verify_credentials.json
YoualsoneedtoobtainaclientIDandclientsecretfromTwitter:
1. CreatearegularaccountatTwitteroruseanexistingaccountyouown.2. VisittheTwitterApplicationManagementdashboardandsigninwithyourTwitteraccount.3. ClickonCreateNewAppandfollowtheinstructionsprovided.TheCallbackURLshouldmatchyouroauth_callbacklater.You
maybepromptedtoaddamobilephonenumbertoyouraccountandverifyit.4. OpentheKeysandAccessTonestab,thennotedowntheConsumerKeyandConsumerSecret.5. SettheoptionclientIdtotheConsumerKeyandtheoptionclientSecrettotheConsumerSecret.
Notethatifyouonlyneedread-onlyaccesstopublicinformation,youcanalsousetheclientIdandclientSecretdirectlywithoutOAuth1.0a.
SeeTwitterRESTAPIReferenceDocumentation.
SettingupOAuth1.0aforXING
IfyouwanttouseXINGastheOAuth1.0aprovider,usethefollowingoptions:
requestTokenEndpoint:https://api.xing.com/v1/request_tokenauthEndpoint:https://api.xing.com/v1/authorizeaccessTokenEndpoint:https://api.xing.com/v1/access_tokenactiveUserEndpoint:https://api.xing.com/v1/users/me
YoualsoneedtoobtainaclientIDandclientsecretfromXING:
1. CreatearegularaccountatXINGoruseanexistingaccountyouown.2. VisittheXINGDeveloperpageandsigninwithyourXINGaccount.3. ClickonCreateappandnotedowntheConsumerkeyandConsumersecret.4. SettheoptionclientIdtotheConsumerkeyandtheoptionclientSecrettotheConsumersecret.
SeeXINGDeveloperDocumentation.
SettingupOAuth1.0aforTumblr
IfyouwanttouseTumblrastheOAuth1.0aprovider,usethefollowingoptions:
requestTokenEndpoint:https://www.tumblr.com/oauth/request_tokenauthEndpoint:https://www.tumblr.com/oauth/authorizeaccessTokenEndpoint:https://www.tumblr.com/oauth/access_tokenactiveUserEndpoint:https://api.tumblr.com/v2/user/info
YoualsoneedtoobtainaclientIDandclientsecretfromTumblr:
1. CreatearegularaccountatTumblroruseanexistingaccountyouown.2. VisittheTumblrApplicationsdashboard.3. ClickonRegisterapplication,thenfollowtheinstructionsprovided.TheDefaultcallbackURLshouldmatchyouroauth_callback
later.4. NotedowntheOAuthConsumerKeyandSecretKey.Thesecretmaybehiddenbydefault.5. SettheoptionclientIdtotheOAuthConsumerKeyandtheoptionclientSecrettotheSecretKey.
SeeTumblrAPIDocumentation.
Fetchanunauthenticatedrequesttokenoauth1.fetchRequestToken(oauth_callback,opts)
Fetchesanoauth_tokenthatcanbeusedtocreateanauthorizationURLthatredirectstothegivenoauth_callbackonconfirmation.
OAuth1.0a
275
PerformsaPOSTresponsetotherequestTokenEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
Arguments
oauth_callback:string
Thefully-qualifiedURLofyourapplication'sOAuth1.0acallback.
opts:Object(optional)
Anobjectwithadditionalqueryparameterstoincludeintherequest.
SeeRFC5849.
Returnstheparsedresponseobject.
GettheauthorizationURLoauth1.getAuthUrl(oauth_token,opts):string
GeneratestheauthorizationURLfortheauthorizationendpoint.
Arguments
oauth_token:string
Theoauth_tokenpreviouslyreturnedbyfetchRequestToken.
opts:(optional)
AnobjectwithadditionalqueryparameterstoaddtotheURL.
SeeRFC5849.
Returnsafully-qualifiedURLfortheauthorizationendpointoftheproviderbyappendingtheoauth_tokenandanyadditionalargumentsfromoptstotheauthEndpoint.
Examples
constrequestToken=oauth1.fetchRequestToken(oauth_callback);
if(requestToken.oauth_callback_confirmed!=='true'){
thrownewError('ProvidercouldnotconfirmOAuth1.0callback');
}
constauthUrl=oauth1.getAuthUrl(requestToken.oauth_token);
Exchangeanauthenticatedrequesttokenforanaccesstokenoauth1.exchangeRequestToken(oauth_token,oauth_verifier,opts)
TakesapairofauthenticatedtemporarycredentialspassedtothecallbackURLbytheproviderandexchangesitforanoauth_tokenandoauth_token_secretthancanbeusedtoperformauthenticatedrequeststotheOAuth1.0aprovider.
PerformsaPOSTresponsetotheaccessTokenEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
Arguments
oauth_token:string
Theoauth_tokenpassedtothecallbackURLbytheprovider.
oauth_verifier:string
Theoauth_verifierpassedtothecallbackURLbytheprovider.
opts:Object(optional)
OAuth1.0a
276
Anobjectwithadditionalqueryparameterstoincludeintherequest.
SeeRFC5849.
Returnstheparsedresponseobject.
Fetchtheactiveuseroauth1.fetchActiveUser(oauth_token,oauth_token_secret,opts):Object
Fetchesdetailsoftheactiveuser.
PerformsaGETresponsetotheactiveUserEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
ReturnsnulliftheactiveUserEndpointisnotconfigured.
Arguments
oauth_token:string
AnOAuth1.0aaccesstokenasreturnedbyexchangeRequestToken.
oauth_token_secret:string
AnOAuth1.0aaccesstokensecretasreturnedbyexchangeRequestToken.
opts:Object(optional)
Anobjectwithadditionalqueryparameterstoincludeintherequest.
SeeRFC5849.
Returnstheparsedresponseobject.
Examples
constauthData=oauth1.exchangeRequestToken(oauth_token,oauth_verifier);
constuserData=oauth1.fetchActiveUser(authData.oauth_token,authData.oauth_token_secret);
Createanauthenticatedrequestobjectoauth1.createSignedRequest(method,url,parameters,oauth_token,oauth_token_secret)
CreatesarequestobjectthatcanbeusedtoperformarequesttotheOAuth1.0aproviderwiththeprovidedtokencredentials.
Arguments
method:string
HTTPmethodtherequestwilluse,e.g."POST".
url:string
Thefully-qualifiedURLoftheprovidertherequestwillbeperformedagainst.
TheURLmayoptionallycontainanynumberofqueryparameters.
parameters:string|Object|null
Anadditionalobjectorquerystringcontainingqueryparametersorbodyparametersthatwillbepartofthesignedrequest.
oauth_token:string
AnOAuth1.0aaccesstokenasreturnedbyexchangeRequestToken.
oauth_token_secret:string
OAuth1.0a
277
AnOAuth1.0aaccesstokensecretasreturnedbyexchangeRequestToken.
Returnsanobjectwiththreeproperties:
url:ThenormalizedURLwithoutanyqueryparameters.
qs:Anormalizedquerystringcontainingallparametersandqueryparameters.
headers:Anobjectcontainingthefollowingproperties:
accept:Thestring"application/json".
authorization:AnOAuthauthorizationheadercontainingallOAuthparametersandtherequestsignature.
Examples
Fetchalistoftweetsmentioning@arangodb:
constrequest=require('@arangodb/request');
constreq=oauth1.createSignedRequest(
'GET',
'https://api.twitter.com/1.1/search/tweets.json',
{q:'@arangodb'},
authData.oauth_token,
authData.oauth_token_secret
);
constres=request(req);
console.log(res.json.statuses);
Signingamorecomplexrequest:
consturl='https://api.example.com/v1/timeline?visible=public';
constparams={hello:'world',longcat:'islong'};
constreq=oauth1.createSignedRequest(
'POST',
url,//URLincludesaqueryparameterthatwillbesigned
params,//Requestbodyneedstobesignedtoo
authData.oauth_token,
authData.oauth_token_secret
);
constres=request.post(url,{
form:params,
headers:{
accept:'application/x-www-form-urlencoded',
//Authorizationheaderincludesthesignature
authorization:req.headers.authorization
}
});
console.log(res.json);
OAuth1.0a
278
OAuth2.0constcreateOAuth2Client=require('@arangodb/foxx/oauth2');
TheOAuth2moduleprovidesabstractionsoverOAuth2.0providerslikeFacebook,GitHubandGoogle.
Examples
Thefollowingextendstheusermanagementexample:
constcrypto=require('@arangodb/crypto');
constrouter=createRouter();
constoauth2=createOAuth2Client({
//We'lluseFacebookforthisexample
authEndpoint:'https://www.facebook.com/dialog/oauth',
tokenEndpoint:'https://graph.facebook.com/oauth/access_token',
activeUserEndpoint:'https://graph.facebook.com/v2.0/me',
clientId:'keyboardcat',
clientSecret:'keyboardcat'
});
module.context.use('/oauth2',router);
//Seetheusermanagementexampleforsettingupthe
//sessionsandusersobjectsusedinthisexample
router.use(sessions);
router.post('/auth',function(req,res){
constcsrfToken=crypto.genRandomAlphaNumbers(32);
consturl=req.reverse('oauth2_callback',{csrfToken});
constredirect_uri=req.makeAbsolute(url);
//SetCSRFcookieforfiveminutes
res.cookie('oauth2_csrf_token',csrfToken,{ttl:60*5});
//Redirecttotheprovider'sauthorizationURL
res.redirect(303,oauth2.getAuthUrl(redirect_uri));
});
router.get('/auth',function(req,res){
//Someproviderspasserrorsasqueryparameter
if(req.queryParams.error){
res.throw(500,`Providererror:${req.queryParams.error}`)
}
//MakesureCSRFcookiematchestheURL
constexpectedToken=req.cookie('oauth2_csrf_token');
if(!expectedToken||req.queryParams.csrfToken!==expectedToken){
res.throw(400,'CSRFmismatch.');
}
//MakesuretheURLcontainsagranttoken
if(!req.queryParams.code){
res.throw(400,'Providerdidnotpassgranttoken.');
}
//Reconstructtheredirect_uriusedforthegranttoken
consturl=req.reverse('oauth2_callback');
constredirect_uri=req.makeAbsolute(url);
//Fetchanaccesstokenfromtheprovider
constauthData=oauth2.exchangeGrantToken(
req.queryParams.code,
redirect_uri
);
constfacebookToken=authData.access_token;
//Fetchtheactiveuser'sprofileinfo
constprofile=oauth2.fetchActiveUser(facebookToken);
constfacebookId=profile.id;
//TrytofindanexistinguserwiththeuserID
//(thisrequirestheuserscollection)
letuser=users.firstExample({facebookId});
if(user){
//UpdatethefacebookTokenifithaschanged
if(user.facebookToken!==facebookToken){
users.update(user,{facebookToken});
}
}else{
OAuth2.0
279
//Createanewuserdocument
user={
username:`fb:${facebookId}`,
facebookId,
facebookToken
}
constmeta=users.save(user);
Object.assign(user,meta);
}
//Logtheuserin(thisrequiresthesessionmiddleware)
req.session.uid=user._key;
req.session.facebookToken=authData.facebookToken;
req.sessionStorage.save(req.session);
//Redirecttothedefaultroute
res.redirect(303,req.makeAbsolute('/'));
},'oauth2_callback')
.queryParam('error',joi.string().optional())
.queryParam('csrfToken',joi.string().optional())
.queryParam('code',joi.string().optional());
CreatinganOAuth2.0clientcreateOAuth2Client(options):OAuth2Client
CreatesanOAuth2.0client.
Arguments
options:Object
Anobjectwiththefollowingproperties:
authEndpoint:string
Thefully-qualifiedURLoftheprovider'sauthorizationendpoint.
tokenEndpoint:string
Thefully-qualifiedURLoftheprovider'stokenendpoint.
refreshEndpoint:string(optional)
Thefully-qualifiedURLoftheprovider'srefreshtokenendpoint.
activeUserEndpoint:string(optional)
Thefully-qualifiedURLoftheprovider'sendpointforfetchingdetailsaboutthecurrentuser.
clientId:string
Theapplication'sClientID(orAppID)fortheprovider.
clientSecret:string
Theapplication'sClientSecret(orAppSecret)fortheprovider.
ReturnsanOAuth2.0clientforthegivenprovider.
SettingupOAuth2.0forFacebook
IfyouwanttouseFacebookastheOAuth2.0provider,usethefollowingoptions:
authEndpoint:https://www.facebook.com/dialog/oauthtokenEndpoint:https://graph.facebook.com/oauth/access_tokenactiveUserEndpoint:https://graph.facebook.com/v2.0/me
YoualsoneedtoobtainaclientIDandclientsecretfromFacebook:
1. CreatearegularaccountatFacebookoruseanexistingaccountyouown.2. VisittheFacebookDeveloperspage.
OAuth2.0
280
3. ClickonAppsinthemenu,thenselectRegisterasaDeveloper(theonlyoption)andfollowtheinstructionsprovided.Youmayneedtoverifyyouraccountbyphone.
4. ClickonAppsinthemenu,thenselectCreateaNewAppandfollowtheinstructionsprovided.5. Opentheappdashboard,thennotedowntheAppIDandAppSecret.Thesecretmaybehiddenbydefault.6. ClickonSettings,thenAdvancedandenteroneormoreValidOAuthredirectURIs.Atleastoneofthemmustmatchyourredirect_urilater.Don'tforgettosaveyourchanges.
7. SettheoptionclientIdtotheAppIDandtheoptionclientSecrettotheAppSecret.
SettingupOAuth2.0forGitHub
IfyouwanttouseGitHubastheOAuth2.0provider,usethefollowingoptions:
authEndpoint:https://github.com/login/oauth/authorize?scope=usertokenEndpoint:https://github.com/login/oauth/access_tokenactiveUserEndpoint:https://api.github.com/user
YoualsoneedtoobtainaclientIDandclientsecretfromGitHub:
1. CreatearegularaccountatGitHuboruseanexistingaccountyouown.2. GotoAccountSettings>Applications>Registernewapplication.3. ProvideanauthorizationcallbackURL.Thismustmatchyourredirect_urilater.4. Fillintheotherrequireddetailsandfollowtheinstructionsprovided.5. Opentheapplicationpage,thennotedowntheClientIDandClientSecret.6. SettheoptionclientIdtotheClientIDandtheoptionclientSecrettotheClientSecret.
SettingupOAuth2.0forGoogle
IfyouwanttouseGoogleastheOAuth2.0provider,usethefollowingoptions:
authEndpoint:https://accounts.google.com/o/oauth2/auth?access_type=offline&scope=profiletokenEndpoint:https://accounts.google.com/o/oauth2/tokenactiveUserEndpoint:https://www.googleapis.com/plus/v1/people/me
YoualsoneedtoobtainaclientIDandclientsecretfromGoogle:
1. CreatearegularaccountatGoogleoruseanexistingaccountyouown.2. VisittheGoogleDevelopersConsole.3. ClickonCreateProject,thenfollowtheinstructionsprovided.4. Whenyourprojectisready,opentheprojectdashboard,thenclickonEnableanAPI.5. EnabletheGoogle+APItoallowyourapptodistinguishbetweendifferentusers.6. OpentheCredentialspageandclickCreatenewClientID,thenfollowtheinstructionsprovided.AtleastoneAuthorizedRedirectURImustmatchyourredirect_urilater.AtleastoneAuthorizedJavaScriptOriginmustmatchyourapp'sfully-qualifieddomain.
7. WhentheClientIDisready,notedowntheClientIDandClientsecret.8. SettheoptionclientIdtotheClientIDandtheoptionclientSecrettotheClientsecret.
GettheauthorizationURLoauth2.getAuthUrl(redirect_uri,args):string
GeneratestheauthorizationURLfortheauthorizationendpoint.
Arguments
redirect_uri:string
Thefully-qualifiedURLofyourapplication'sOAuth2.0callback.
args:(optional)
Anobjectwithanyofthefollowingproperties:
response_type:string(Default:"code")
OAuth2.0
281
SeeRFC6749.
Returnsafully-qualifiedURLfortheauthorizationendpointoftheproviderbyappendingtheclientIDandanyadditionalargumentsfromargstotheauthEndpoint.
Exchangeagrantcodeforanaccesstokenoauth2.exchangeGrantToken(code,redirect_uri)
Exchangesagrantcodeforanaccesstoken.
PerformsaPOSTresponsetothetokenEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
Arguments
code:string
Agrantcodereturnedbytheprovider'sauthorizationendpoint.
redirect_uri:string
TheoriginalcallbackURLwithwhichthecodewasrequested.
args:Object(optional)
Anobjectwithanyofthefollowingproperties:
grant_type:string(Default:"authorization_code")
SeeRFC6749.
Returnstheparsedresponseobject.
Fetchtheactiveuseroauth2.fetchActiveUser(access_token):Object
Fetchesdetailsoftheactiveuser.
PerformsaGETresponsetotheactiveUserEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
ReturnsnulliftheactiveUserEndpointisnotconfigured.
Arguments
access_token:string
AnOAuth2.0accesstokenasreturnedbyexchangeGrantToken.
Returnstheparsedresponseobject.
Examples
constauthData=oauth2.exchangeGrantToken(code,redirect_uri);
constuserData=oauth2.fetchActiveUser(authData.access_token);
OAuth2.0
282
TransactionsStartingwithversion1.3,ArangoDBprovidessupportforuser-definabletransactions.
TransactionsinArangoDBareatomic,consistent,isolated,anddurable(ACID).
TheseACIDpropertiesprovidethefollowingguarantees:
Theatomicityprinciplemakestransactionseithercompleteintheirentiretyorhavenoeffectatall.Theconsistencyprincipleensuresthatnoconstraintsorotherinvariantswillbeviolatedduringorafteranytransaction.Theisolationpropertywillhidethemodificationsofatransactionfromothertransactionsuntilthetransactioncommits.Finally,thedurabilitypropositionmakessurethatoperationsfromtransactionsthathavecommittedwillbemadepersistent.TheamountoftransactiondurabilityisconfigurableinArangoDB,asisthedurabilityoncollectionlevel.
Transactions
283
TransactioninvocationArangoDBtransactionsaredifferentfromtransactionsinSQL.
InSQL,transactionsarestartedwithexplicitBEGINorSTARTTRANSACTIONcommand.Followinganyseriesofdataretrievalormodificationoperations,anSQLtransactionisfinishedwithaCOMMITcommand,orrolledbackwithaROLLBACKcommand.Theremaybeclient/servercommunicationbetweenthestartandthecommit/rollbackofanSQLtransaction.
InArangoDB,atransactionisalwaysaserver-sideoperation,andisexecutedontheserverinonego,withoutanyclientinteraction.Alloperationstobeexecutedinsideatransactionneedtobeknownbytheserverwhenthetransactionisstarted.
TherearenoindividualBEGIN,COMMITorROLLBACKtransactioncommandsinArangoDB.Instead,atransactioninArangoDBisstartedbyprovidingadescriptionofthetransactiontothedb._executeTransactionJavaScriptfunction:
db._executeTransaction(description);
Thisfunctionwillthenautomaticallystartatransaction,executeallrequireddataretrievaland/ormodificationoperations,andattheendautomaticallycommitthetransaction.Ifanerroroccursduringtransactionexecution,thetransactionisautomaticallyaborted,andallchangesarerolledback.
Executetransaction
executesatransactiondb._executeTransaction(object)
Executesaserver-sidetransaction,asspecifiedbyobject.
objectmusthavethefollowingattributes:
collections:asub-objectthatdefineswhichcollectionswillbeusedinthetransaction.collectionscanhavetheseattributes:read:asinglecollectionoralistofcollectionsthatwillbeusedinthetransactioninread-onlymodewrite:asinglecollectionoralistofcollectionsthatwillbeusedinthetransactioninwriteorreadmode.
action:aJavascriptfunctionorastringwithJavascriptcodecontainingalltheinstructionstobeexecutedinsidethetransaction.Ifthecoderunsthroughsuccessfully,thetransactionwillbecommittedattheend.Ifthecodethrowsanexception,thetransactionwillberolledbackandalldatabaseoperationswillberolledback.
Additionally,objectcanhavethefollowingoptionalattributes:
waitForSync:booleanflagindicatingwhetherthetransactionisforcedtobesynchronous.lockTimeout:anumericvaluethatcanbeusedtosetatimeoutforwaitingoncollectionlocks.Ifnotspecified,adefaultvaluewillbeused.SettinglockTimeoutto0willmakeArangoDBnottimeoutwaitingforalock.params:optionalargumentspassedtothefunctionspecifiedinaction.
ThefollowingattributescanbeusedfortransactionsintheRocksDBstorageengine:
maxTransactionSize:transactionsizelimitinbytesintermediateCommitSize:maximumtotalsizeofoperationsafterwhichanintermediatecommitisperformedautomaticallyintermediateCommitCount:maximumnumberofoperationsafterwhichanintermediatecommitisperformedautomatically
Declarationofcollections
Allcollectionswhicharetoparticipateinatransactionneedtobedeclaredbeforehand.Thisisanecessitytoensureproperlockingandisolation.
Collectionscanbeusedinatransactioninwritemodeorinread-onlymode.
Ifanydatamodificationoperationsaretobeexecuted,thecollectionmustbedeclaredforuseinwritemode.Thewritemodeallowsmodifyingandreadingdatafromthecollectionduringthetransaction(i.e.thewritemodeincludesthereadmode).
Contrary,usingacollectioninread-onlymodewillonlyallowperformingreadoperationsonacollection.Anyattempttowriteintoacollectionusedinread-onlymodewillmakethetransactionfail.
Transactioninvocation
284
Collectionsforatransactionaredeclaredbyprovidingtheminthecollectionsattributeoftheobjectpassedtothe_executeTransactionfunction.Thecollectionsattributehasthesub-attributesreadandwrite:
db._executeTransaction({
collections:{
write:["users","logins"],
read:["recommendations"]
}
});
readandwriteareoptionalattributes,andonlyneedtobespecifiediftheoperationsinsidethetransactionsdemandforit.
Thecontentsofreadorwritecaneachbelistsarrayscollectionnamesorasinglecollectionname(asastring):
db._executeTransaction({
collections:{
write:"users",
read:"recommendations"
}
});
Note:Itiscurrentlyoptionaltospecifycollectionsforread-onlyaccess.Evenwithoutspecifyingthem,itisstillpossibletoreadfromsuchcollectionsfromwithinatransaction,butwithrelaxedisolation.PleaserefertoTransactionsLockingformoredetails.
Inordertomakeatransactionfailwhenanon-declaredcollectionisusedinsideforreading,theoptionalallowImplicitsub-attributeofcollectionscanbesettofalse:
db._executeTransaction({
collections:{
read:"recommendations",
allowImplicit:false/*thisdisallowsreadaccesstoothercollections
thanspecified*/
},
action:function(){
vardb=require("@arangodb").db;
returndb.foobar.toArray();/*willfailbecausedb.foobarmustnotbeaccessed
forreadinginsidethistransaction*/
}
});
ThedefaultvalueforallowImplicitistrue.Write-accessingcollectionsthathavenotbeendeclaredinthecollectionsarrayisneverpossible,regardlessofthevalueofallowImplicit.
Declarationofdatamodificationandretrievaloperations
AlldatamodificationandretrievaloperationsthataretobeexecutedinsidethetransactionneedtobespecifiedinaJavascriptfunction,usingtheactionattribute:
db._executeTransaction({
collections:{
write:"users"
},
action:function(){
//alloperationsgohere
}
});
AnyvalidJavascriptcodeisallowedinsideactionbutthecodemayonlyaccessthecollectionsdeclaredincollections.actionmaybeaJavascriptfunctionasshownabove,orastringrepresentationofaJavascriptfunction:
db._executeTransaction({
collections:{
write:"users"
},
action:"function(){doSomething();}"
});
Transactioninvocation
285
Pleasenotethatanyoperationsspecifiedinactionwillbeexecutedontheserver,inaseparatescope.Variableswillbeboundlate.AccessinganyJavaScriptvariablesdefinedontheclient-sideorinsomeotherservercontextfrominsideatransactionmaynotwork.Instead,anyvariablesusedinsideactionshouldbedefinedinsideactionitself:
db._executeTransaction({
collections:{
write:"users"
},
action:function(){
vardb=require(...).db;
db.users.save({...});
}
});
Whenthecodeinsidetheactionattributeisexecuted,thetransactionisalreadystartedandallrequiredlockshavebeenacquired.Whenthecodeinsidetheactionattributefinishes,thetransactionwillautomaticallycommit.Thereisnoexplicitcommitcommand.
Tomakeatransactionabortandrollbackallchanges,anexceptionneedstobethrownandnotcaughtinsidethetransaction:
db._executeTransaction({
collections:{
write:"users"
},
action:function(){
vardb=require("@arangodb").db;
db.users.save({_key:"hello"});
//willabortandrollbackthetransaction
throw"doh!";
}
});
Thereisnoexplicitabortorrollbackcommand.
Asmentionedearlier,atransactionwillcommitautomaticallywhentheendoftheactionfunctionisreachedandnoexceptionhasbeenthrown.Inthiscase,theusercanreturnanylegalJavaScriptvaluefromthefunction:
db._executeTransaction({
collections:{
write:"users"
},
action:function(){
vardb=require("@arangodb").db;
db.users.save({_key:"hello"});
//willcommitthetransactionandreturnthevalue"hello"
return"hello";
}
});
Customexceptions
Onemaywishtodefinecustomexceptionsinsideofatransaction.Tohavetheexceptionpropagateupwardsproperly,pleasethrowananinstanceofbaseJavaScriptErrorclassoraderivative.Tospecifyanerrornumber,includeitastheerrorNumberfield.Asanexample:
db._executeTransaction({
collections:{},
action:function(){
varerr=newError('Myerrorcontext');
err.errorNumber=1234;
throwerr;
}
});
Transactioninvocation
286
Note:Inpreviousversions,customexceptionswhichdidnothaveanError-likeformweresimplyconvertedtostringsandexposedintheexceptionfieldofthereturnederror.Thisisnolongerthecase,asithadthepotentialtoleakunwantedinformationifimproperlyused.
Examples
Thefirstexamplewillwrite3documentsintoacollectionnamedc1.Thec1collectionneedstobedeclaredinthewriteattributeofthecollectionsattributepassedtotheexecuteTransactionfunction.
Theactionattributecontainstheactualtransactioncodetobeexecuted.Thiscodecontainsalldatamodificationoperations(3inthisexample).
//setup
db._create("c1");
db._executeTransaction({
collections:{
write:["c1"]
},
action:function(){
vardb=require("@arangodb").db;
db.c1.save({_key:"key1"});
db.c1.save({_key:"key2"});
db.c1.save({_key:"key3"});
}
});
db.c1.count();//3
Abortingthetransactionbythrowinganexceptionintheactionfunctionwillrevertallchanges,soasifthetransactionneverhappened:
//setup
db._create("c1");
db._executeTransaction({
collections:{
write:["c1"]
},
action:function(){
vardb=require("@arangodb").db;
db.c1.save({_key:"key1"});
db.c1.count();//1
db.c1.save({_key:"key2"});
db.c1.count();//2
throw"doh!";
}
});
db.c1.count();//0
Theautomaticrollbackisalsoexecutedwhenaninternalexceptionisthrownatsomepointduringtransactionexecution:
//setup
db._create("c1");
db._executeTransaction({
collections:{
write:["c1"]
},
action:function(){
vardb=require("@arangodb").db;
db.c1.save({_key:"key1"});
//willthrowduplicateakeyerror,notexplicitlyrequestedbytheuser
db.c1.save({_key:"key1"});
//we'llnevergethere...
}
});
db.c1.count();//0
Transactioninvocation
287
Asrequiredbytheconsistencyprinciple,abortingorrollingbackatransactionwillalsorestoresecondaryindexestothestateattransactionstart.
Cross-collectiontransactions
There'salsothepossibilitytorunatransactionacrossmultiplecollections.Inthiscase,multiplecollectionsneedtobedeclaredinthecollectionsattribute,e.g.:
//setup
db._create("c1");
db._create("c2");
db._executeTransaction({
collections:{
write:["c1","c2"]
},
action:function(){
vardb=require("@arangodb").db;
db.c1.save({_key:"key1"});
db.c2.save({_key:"key2"});
}
});
db.c1.count();//1
db.c2.count();//1
Again,throwinganexceptionfrominsidetheactionfunctionwillmakethetransactionabortandrollbackallchangesinallcollections:
//setup
db._create("c1");
db._create("c2");
db._executeTransaction({
collections:{
write:["c1","c2"]
},
action:function(){
vardb=require("@arangodb").db;
for(vari=0;i<100;++i){
db.c1.save({_key:"key"+i});
db.c2.save({_key:"key"+i});
}
db.c1.count();//100
db.c2.count();//100
//abort
throw"doh!"
}
});
db.c1.count();//0
db.c2.count();//0
Transactioninvocation
288
PassingparameterstotransactionsArbitraryparameterscanbepassedtotransactionsbysettingtheparamsattributewhendeclaringthetransaction.Thisfeatureishandytore-usethesametransactioncodeformultiplecallsbutwithdifferentparameters.
Abasicexample:
db._executeTransaction({
collections:{},
action:function(params){
returnparams[1];
},
params:[1,2,3]
});
Theaboveexamplewillreturn2.
Someexamplethatusescollections:
db._executeTransaction({
collections:{
write:"users",
read:["c1","c2"]
},
action:function(params){
vardb=require('@arangodb').db;
vardoc=db.c1.document(params['c1Key']);
db.users.save(doc);
doc=db.c2.document(params['c2Key']);
db.users.save(doc);
},
params:{
c1Key:"foo",
c2Key:"bar"
}
});
Passingparameters
289
LockingandIsolationTransactionsneedtospecifyfromwhichcollectionstheywillreaddataandwhichcollectionstheyintenddomodify.Thiscanbedonebysettingtheread,write,orexclusiveattributesinthecollectionsattributeofthetransaction:
db._executeTransaction({
collections:{
read:"users",
write:["test","log"]
},
action:function(){
constdb=require("@arangodb").db;
db.users.toArray().forEach(function(doc){
db.log.insert({value:"removeduser:"+doc.name});
db.test.remove(doc._key);
});
}
});
writeheremeanswriteaccesstothecollection,andalsoincludesanyreadaccesses.exclusiveisasynonymforwriteintheMMFilesengine,becausebothexclusiveandwritewillacquirecollection-levellocksinthisengine.IntheRocksDBengine,exclusivemeansexclusivewriteaccesstothecollection,andwritemeans(shared)writeaccesstothecollection,whichcanbeinterleavedwithwriteaccessesbyotherconcurrenttransactions.
MMFilesengine
TheMMFilesengineusesthefollowinglockingmechanismstoserializetransactionsonthesamedata:
Allcollectionsspecifiedinthecollectionsattributearelockedintherequestedmode(readorwrite)attransactionstart.Lockingofmultiplecollectionsisperformedinalphabeticalorder.Whenatransactioncommitsorrollsback,alllocksarereleasedinreverseorder.Thelockingorderisdeterministictoavoiddeadlocks.
Whilelocksareheld,modificationsbyothertransactionstothecollectionsparticipatinginthetransactionareprevented.Atransactionwillthusseeaconsistentviewoftheparticipatingcollections'data.
Additionally,atransactionwillnotbeinterruptedorinterleavedwithanyotherongoingoperationsonthesamecollection.Thismeanseachtransactionwillruninisolation.Atransactionshouldneverseeuncommittedorrolledbackmodificationsbyothertransactions.Additionally,readsinsideatransactionarerepeatable.
Notethattheaboveistrueonlyforallcollectionsthataredeclaredinthecollectionsattributeofthetransaction.
RocksDBengine
TheRocksDBenginedoesnotlockanycollectionsparticipatinginatransactionforread.Readoperationscanruninparalleltootherreadorwriteoperationsonthesamecollections.
Forallcollectionsthatareusedinwritemode,theRocksDBenginewillinternallyacquirea(shared)readlock.Thismeansthatmanywriterscanmodifydatainthesamecollectioninparallel(andalsoruninparalleltoongoingreads).However,iftwoconcurrenttransactionsattempttomodifythesamedocumentorindexentry,therewillbeawrite-writeconflict,andoneofthetransactionswillabortwitherror1200("conflict").Itisthenuptoclientapplicationstoretrythefailedtransactionoracceptthefailure.
Inordertoguardlong-runningorcomplextransactionsagainstconcurrentoperationsonthesamedata,theRocksDBengineallowstoaccesscollectionsinexclusivemode.Exclusiveaccesseswillinternallyacquireawrite-lockonthecollections,sotheyarenotexecutedinparallelwithanyotherwriteoperations.Readoperationscanstillbecarriedoutbyotherconcurrenttransactions.
Lazilyaddingcollections
Theremightbesituationswhendeclaringallcollectionsaprioriisnotpossible,forexample,becausefurthercollectionsaredeterminedbyadynamicAQLqueryinsidethetransaction,forexampleaqueryusingAQLgraphtraversal.
Lockingandisolation
290
Inthiscase,itwouldbeimpossibletoknowbeforehandwhichcollectiontolock,andthusitislegaltonotdeclarecollectionsthatwillbeaccessedinthetransactioninread-onlymode.Accessinganon-declaredcollectioninread-onlymodeduringatransactionwilladdthecollectiontothetransactionlazily,andfetchdatafromthecollectionasusual.However,asthecollectionisaddedlazily,thereisnoisolationfromotherconcurrentoperationsortransactions.Readsfromsuchcollectionsarepotentiallynon-repeatable.
Examples:
db._executeTransaction({
collections:{
read:"users"
},
action:function(){
constdb=require("@arangodb").db;
/*ExecuteanAQLquerythattraversesagraphstartingata"users"vertex.
Itisyetunknownintowhichothercollectionsthequerymighttraverse*/
db._createStatement({
query:`FORvINANY"users/1234"connectionsRETURNv`
}).execute().toArray().forEach(function(d){
/*...*/
});
}
});
Thisautomaticlazyadditionofcollectionstoatransactionalsointroducesthepossibilityofdeadlocks.Deadlocksmayoccurifthereareconcurrenttransactionsthattrytoacquirelocksonthesamecollectionslazily.
Inordertomakeatransactionfailwhenanon-declaredcollectionisusedinsideatransactionforreading,theoptionalallowImplicitsub-attributeofcollectionscanbesettofalse:
db._executeTransaction({
collections:{
read:"users",
allowImplicit:false
},
action:function(){
/*Thebelowquerywillnowfailbecausethecollection"connections"hasnot
beenspecifiedinthelistofcollectionsusedbythetransaction*/
constdb=require("@arangodb").db;
db._createStatement({
query:`FORvINANY"users/1234"connectionsRETURNv`
}).execute().toArray().forEach(function(d){
/*...*/
});
}
});
ThedefaultvalueforallowImplicitistrue.Write-accessingcollectionsthathavenotbeendeclaredinthecollectionsarrayisneverpossible,regardlessofthevalueofallowImplicit.
Ifusers/1234hasanedgeinconnections,linkingittoanotherdocumentintheuserscollection,thenthefollowingexplicitdeclarationwillwork:
db._executeTransaction({
collections:{
read:["users","connections"],
allowImplicit:false
},
/*...*/
Iftheedgepointstoadocumentinanothercollectionhowever,thenthequerywillfail,unlessthatothercollectionisaddedtothedeclarationaswell.
Notethatifadocumenthandleisusedasstartingpointforatraversal,e.g.FORvINANY"users/not_linked"...orFORvINANY{_id:"users/not_linked"}...,thennoerrorisraisedinthecaseofthestartvertexnothavinganyedgestofollow,withallowImplicit:falseandusersnotbeingdeclaredforreadaccess.AQLonlyseesastringanddoesnotconsideritareadaccess,unlessthereareedgesconnectedtoit.FORvINANYDOCUMENT("users/not_linked")...willfailevenwithoutedges,asitisalwaysconsideredtobeareadaccesstotheuserscollection.
Lockingandisolation
291
DeadlocksandDeadlockdetection
Adeadlockisasituationinwhichtwoormoreconcurrentoperations(usertransactionsorAQLqueries)trytoaccessthesameresources(collections,documents)andneedtowaitfortheotherstofinish,butnoneofthemcanmakeanyprogress.
AgoodexampleforadeadlockistwoconcurrentlyexecutingtransactionsT1andT2thattrytoaccessthesamecollectionsbutthatneedtowaitforeachother.Inthisexample,transactionT1willwritetocollectionc1,butwillalsoreaddocumentsfromcollectionc2withoutannouncingit:
db._executeTransaction({
collections:{
write:"c1"
},
action:function(){
constdb=require("@arangodb").db;
/*writeintoc1(announced)*/
db.c1.insert({foo:"bar"});
/*someoperationherethattakeslongtoexecute...*/
/*readfromc2(unannounced)*/
db.c2.toArray();
}
});
TransactionT2announcestowriteintocollectionc2,butwillalsoreaddocumentsfromcollectionc1withoutannouncingit:
db._executeTransaction({
collections:{
write:"c2"
},
action:function(){
vardb=require("@arangodb").db;
/*writeintoc2(announced)*/
db.c2.insert({bar:"baz"});
/*someoperationherethattakeslongtoexecute...*/
/*readfromc1(unannounced)*/
db.c1.toArray();
}
});
Intheaboveexample,adeadlockwilloccuriftransactionT1andT2havebothacquiredtheirwritelocks(T1forcollectionc1andT2forcollectionc2)andarethentryingtoreadfromtheotherother(T1willreadfromc2,T2willreadfromc1).T1willthentrytoacquirethereadlockoncollectionc2,whichispreventedbytransactionT2.T2howeverwillwaitforthereadlockoncollectionc1,whichispreventedbytransactionT1.
Incaseofsuchdeadlock,therewouldbenoprogressforanyoftheinvolvedtransactions,andnoneoftheinvolvedtransactionscouldevercomplete.Thisiscompletelyundesirable,sotheautomaticdeadlockdetectionmechanisminArangoDBwillautomaticallyabortoneofthetransactionsinvolvedinsuchdeadlock.Abortingmeansthatallchangesdonebythetransactionwillberolledbackanderror29(deadlockdetected)willbethrown.
Clientcode(AQLqueries,usertransactions)thataccessesmorethanonecollectionshouldbeawareofthepotentialofdeadlocksandshouldhandletheerror29(deadlockdetected)properly,eitherbypassingtheexceptiontothecallerorretryingtheoperation.
Toavoidbothdeadlocksandnon-repeatablereads,allcollectionsusedinatransactionshouldbespecifiedinthecollectionsattributewhenknowninadvance.Incasethisisnotpossiblebecausecollectionsareaddeddynamicallyinsidethetransaction,deadlocksmayoccurandthedeadlockdetectionmaykickinandabortthetransaction.
TheRocksDBengineusesdocument-levellocksandthereforewillnothaveadeadlockproblemoncollectionlevel.Iftwoconcurrenttransactionshowevermodifythesamedocumentsorindexentries,theRocksDBenginewillsignalawrite-writeconflictandabortoneofthetransactionswitherror1200("conflict")automatically.
Lockingandisolation
292
Lockingandisolation
293
DurabilityTransactionsareexecutedinmainmemoryfirstuntilthereiseitherarollbackoracommit.Onrollback,nodatawillbewrittentodisk,buttheoperationsfromthetransactionwillbereversedinmemory.
Oncommit,allmodificationsdoneinthetransactionwillbewrittentothecollectiondatafiles.ThesewriteswillbesynchronizedtodiskifanyofthemodifiedcollectionshasthewaitForSyncpropertysettotrue,orifanyindividualoperationinthetransactionwasexecutedwiththewaitForSyncattribute.Additionally,transactionsthatmodifydatainmorethanonecollectionareautomaticallysynchronizedtodisk.Thissynchronizationisdonetonotonlyensuredurability,buttoalsoensureconsistencyincaseofaservercrash.
Thatmeansifyouonlymodifydatainasinglecollection,andthatcollectionhasitswaitForSyncpropertysettofalse,thewholetransactionwillnotbesynchronizedtodiskinstantly,butwithasmalldelay.
Thereisthusthepotentialriskoflosingdatabetweenthecommitofthetransactionandtheactual(delayed)disksynchronization.ThisisthesameaswritingintocollectionsthathavethewaitForSyncpropertysettofalseoutsideofatransaction.IncaseofacrashwithwaitForSyncsettofalse,theoperationsperformedinthetransactionwilleitherbevisiblecompletelyornotatall,dependingonwhetherthedelayedsynchronizationhadkickedinornot.
ToensuredurabilityoftransactionsonacollectionthathavethewaitForSyncpropertysettofalse,youcansetthewaitForSyncattributeoftheobjectthatispassedtoexecuteTransaction.ThiswillforceasynchronizationofthetransactiontodiskevenforcollectionsthathavewaitForSyncsettofalse:
db._executeTransaction({
collections:{
write:"users"
},
waitForSync:true,
action:function(){...}
});
Analternativeistoperformanoperationwithanexplicitsyncrequestinatransaction,e.g.
db.users.save({_key:"1234"},true);
Inthiscase,thetruevaluewillmakethewholetransactionbesynchronizedtodiskatthecommit.
Inanycase,ArangoDBwillgiveusersthechoiceofwhetherornottheywantfulldurabilityforsinglecollectiontransactions.Usingthedelayedsynchronization(i.e.waitForSyncwithavalueoffalse)willpotentiallyincreasethroughputandperformanceoftransactions,butwillintroducetheriskoflosingthelastcommittedtransactionsinthecaseofacrash.
Incontrast,transactionsthatmodifydatainmorethanonecollectionareautomaticallysynchronizedtodisk.Thiscomesatthecostofseveraldisksync.Foramulti-collectiontransaction,thecalltothe_executeTransactionfunctionwillonlyreturnafterthedataofallmodifiedcollectionshasbeensynchronizedtodiskandthetransactionhasbeenmadefullydurable.Thisnotonlyreducestheriskoflosingdataincaseofacrashbutalsoensuresconsistencyafterarestart.
Incaseofaservercrash,anymulti-collectiontransactionsthatwerenotyetcommittedorinpreparationtobecommittedwillberolledbackonserverrestart.
Formulti-collectiontransactions,therewillbeatleastonedisksyncoperationpermodifiedcollection.Multi-collectiontransactionsthushaveapotentiallyhighercostthansinglecollectiontransactions.Thereisnoconfigurationtoturnoffdisksynchronizationformulti-collectiontransactionsinArangoDB.Thedisksyncspeedofthesystemwillthusbethemostimportantfactorfortheperformanceofmulti-collectiontransactions.
Durability
294
Limitations
InGeneral
TransactionsinArangoDBhavebeendesignedwithparticularusecasesinmind.Theywillbemainlyusefulforshortandsmalldataretrievaland/ormodificationoperations.
Theimplementationisnotoptimizedforverylong-runningorveryvoluminousoperations,andmaynotbeusableforthesecases.
Onelimitationisthatatransactionoperationinformationmustfitintomainmemory.Thetransactioninformationconsistsofrecordpointers,revisionnumbersandrollbackinformation.Theactualdatamodificationoperationsofatransactionarewrittentothewrite-aheadloganddonotneedtofitentirelyintomainmemory.
Ongoingtransactionswillalsopreventthewrite-aheadlogsfrombeingfullygarbage-collected.Informationinthewrite-aheadlogfilescannotbewrittentocollectiondatafilesorbediscardedwhiletransactionsareongoing.
Toensureprogressofthewrite-aheadloggarbagecollection,transactionsshouldbekeptassmallaspossible,andbigtransactionsshouldbesplitintomultiplesmallertransactions.
TransactionsinArangoDBcannotbenested,i.e.atransactionmustnotstartanothertransaction.Ifanattemptismadetocallatransactionfrominsidearunningtransaction,theserverwillthrowerror1651(nestedtransactionsdetected).
ItisalsodisallowedtoexecuteusertransactiononsomeofArangoDB'sownsystemcollections.Thisshouldn'tbeaproblemforregularusageassystemcollectionswillnotcontainuserdataandthereisnoneedtoaccessthemfromwithinausertransaction.
Someoperationsarenotallowedinsidetransactionsingeneral:
creationanddeletionofdatabases(db._createDatabase(),db._dropDatabase())creationanddeletionofcollections(db._create(),db._drop(),db.<collection>.rename())creationanddeletionofindexes(db.<collection>.ensureIndex(),db.<collection>.dropIndex())
Ifanattemptismadetocarryoutanyoftheseoperationsduringatransaction,ArangoDBwillabortthetransactionwitherrorcode1653(disallowedoperationinsidetransaction).
Finally,allcollectionsthatmaybemodifiedduringatransactionmustbedeclaredbeforehand,i.e.usingthecollectionsattributeoftheobjectpassedtothe_executeTransactionfunction.Ifanyattemptismadetocarryoutadatamodificationoperationonacollectionthatwasnotdeclaredinthecollectionsattribute,thetransactionwillbeabortedandArangoDBwillthrowerror1652unregisteredcollectionusedintransaction.Itislegaltonotdeclareread-onlycollections,butthisshouldbeavoidedifpossibletoreducetheprobabilityofdeadlocksandnon-repeatablereads.
PleaserefertoLockingandIsolationformoredetails.
InClustersUsingasingleinstanceofArangoDB,multi-document/multi-collectionqueriesareguaranteedtobefullyACID.ThisismorethanmanyotherNoSQLdatabasesystemssupport.Inclustermode,single-documentoperationsarealsofullyACID.Multi-document/multi-collectionqueriesinaclusterarenotACID,whichisequallythecasewithcompetingdatabasesystems.TransactionsinaclusterwillbesupportedinafutureversionofArangoDBandmaketheseoperationsfullyACIDaswell.Notethatfornon-shardedcollectionsinacluster,thetransactionalpropertiesofasingleserverapply(fullyACID).
TransactionsintheRocksDBstorageengineDataofongoingtransactionsisstoredinRAM.Transactionsthatgettoobig(intermsofnumberofoperationsinvolvedorthetotalsizeofdatacreatedormodifiedbythetransaction)willbecommittedautomatically.EffectivelythismeansthatbigusertransactionsaresplitintomultiplesmallerRocksDBtransactionsthatarecommittedindividually.TheentireusertransactionwillnotnecessarilyhaveACIDpropertiesinthiscase.
ThefollowingglobaloptionscanbeusedtocontroltheRAMusageandautomaticintermediatecommitsfortheRocksDBengine:
Limitations
295
--rocksdb.max-transaction-size
Transactionsizelimit(inbytes).TransactionsstoreallkeysandvaluesinRAM,solargetransactionsruntheriskofcausingout-of-memorysitations.Thissettingallowsyoutoensurethatdoesnothappenbylimitingthesizeofanyindividualtransaction.TransactionswhoseoperationswouldconsumemoreRAMthanthisthresholdvaluewillabortautomaticallywitherror32("resourcelimitexceeded").
--rocksdb.intermediate-commit-size
Ifthesizeofalloperationsinatransactionreachesthisthreshold,thetransactioniscommittedautomaticallyandanewtransactionisstarted.Thevalueisspecifiedinbytes.
--rocksdb.intermediate-commit-count
Ifthenumberofoperationsinatransactionreachesthisvalue,thetransactioniscommittedautomaticallyandanewtransactionisstarted.
Theabovevaluescanalsobeadjustedpertransaction,bysettingthefollowingattributesinthecalltodb._executeTransaction():
maxTransactionSize:transactionsizelimitinbytesintermediateCommitSize:maximumtotalsizeofoperationsafterwhichanintermediatecommitisperformedautomaticallyintermediateCommitCount:maximumnumberofoperationsafterwhichanintermediatecommitisperformedautomatically
Limitations
296
DeploymentInthischapterwedescribevariouspossibilitiestodeployArangoDB.Inparticularfortheclustermode,therearedifferentwaysandwewanttohighlighttheiradvantagesanddisadvantages.Weevendocumentindetail,howtosetupaclusterbysimplystartingvariousArangoDBprocessesondifferentmachines,eitherdirectlyorusingDockercontainers.
SingleinstanceCluster:DC/OS,ApacheMesosandMarathonCluster:Generic&DockerMultipleDatacentersAdvancedTopics
Cluster:TestsetuponalocalmachineCluster:StartingprocessesondifferentmachinesCluster:LaunchinganArangoDBclusterusingDockercontainersAgency
Deployment
297
Singleinstancedeployment
ThelatestofficialbuildsofArangoDBforallsupportedoperatingsystemsmaybeobtainedfromhttps://www.arangodb.com/download/.
Linuxremarks
Besidestheofficialimageswhichareprovidedforthemostpopularlinuxdistributionstherearealsoavarietyofunofficialimagesprovidedbythecommunity.Wearetrackingmostofthecommunitycontributions(includingneworupdatedimages)inournewsletter:
https://www.arangodb.com/category/newsletter/
Windowsremarks
PleasenotethatArangoDBwillonlyworkon64bit.
Docker
ThesimplestwaytodeployArangoDBisusingDocker.TogetageneralunderstandingofDockerhavealookattheirexcellentdocumentation.
Authentication
TostarttheofficialDockercontaineryouwillhavetodecideonanauthenticationmethod.Otherwisethecontainerwon'tstart.
ProvideoneoftheargumentstoDockerasanenvironmentvariable.
Therearethreeoptions:
1. ARANGO_NO_AUTH=1
Disableauthenticationcompletely.Usefulforlocaltestingorforoperatinginatrustednetwork(withoutapublicinterface).
2. ARANGO_ROOT_PASSWORD=password
StartArangoDBwiththegivenpasswordforroot
3. ARANGO_RANDOM_ROOT_PASSWORD=1
LetArangoDBgeneratearandomrootpassword
Togetgoingquickly:
dockerrun-eARANGO_RANDOM_ROOT_PASSWORD=1arangodb/arangodb
ForanindepthguideaboutDockerandArangoDBpleasechecktheofficialdocumentation:https://hub.docker.com/r/arangodb/arangodb/.Notethatweareusingtheimagearangodb/arangodbherewhichisalwaysthemostcurrentone.Thereisalsothe"official"onecalledarangodbwhosedocumentationishere:https://hub.docker.com/_/arangodb/
Singleinstance
298
DistributeddeploymentusingApacheMesos
ArangoDBhasasophisticatedandyeteasywaytouseclustermode.Toleveragethefullclusterfeatureset(monitoring,scaling,automaticfailoverandautomaticreplacementoffailednodes)youhavetorunArangoDBonsomekindofclustermanagementsystem.CurrentlyArangoDBreliesonApacheMesosinthatmatter.Mesosisaclusteroperatingsystemwhichpowerssomeoftheworldsbiggestdatacentersrunningseveralthousandsofnodes.
DC/OS
DC/OSistherecommendedwaytoinstallaclusterasiteasesmuchoftheprocesstoinstallaMesoscluster.YoucandeployitveryquicklyonavarietyofcloudhostersorsetupyourownDC/OSlocally.DC/OSisasetoftoolsbuiltontopofApacheMesos.ApacheMesosisasocalled"DistributedClusterOperationSystem"andthecoreofDC/OS.ApacheMesoshastheconceptofsocalledpersistentvolumeswhichmakeitperfectlysuitableforadatabase.
Installing
FirstprepareaDC/OSclusterbygoingtohttps://dcos.ioandfollowingtheinstructionsthere.
DC/OScomeswithitsownpackagemanagement.Packagescanbeinstalledfromthesocalled"Universe".AsanofficialDC/OSpartnerArangoDBcanbeinstalledfromtherestraightaway.
1. InstallingviaDC/OSUI
i. OpenyourbrowserandgototheDC/OSadmininterfaceii. Openthe"Universe"tabiii. Locatearangodbandhit"InstallPackage"iv. Press"InstallPackage"
2. InstallingviatheDC/OScommandline
i. Installthedcoscliii. Openaterminalandissuedcosinstallarangodb
Bothoptionsareessentiallydoingthesameinthebackground.BotharestartingArangoDBwithitsdefaultoptionsset.
Toreviewthedefaultoptionsusingthewebinterfacesimplyclick"AdvancedInstallation"inthewebinterface.Thereyouwillfindalistofoptionsincludingsomeexplanation.
ToreviewthedefaultoptionsusingtheCLIfirsttypedcospackagedescribe--configarangodb.Thiswillgiveyouaflatlistofdefaultsettings.
Togetanexplanationofthevariouscommandlineoptionspleasecheckthelatestoptionshere(choosethemostrecentnumberandhavealookatconfig.json):
https://github.com/mesosphere/universe/tree/version-3.x/repo/packages/A/arangodb
AfterinstallationDC/OSwillstartdeployingtheArangoDBclusterontheDC/OScluster.YoucanwatchArangoDBstartingonthe"Services"tabinthewebinterface.OnceitislistedashealthyclickthelinknexttoitandyoushouldseetheArangoDBwebinterface.
ArangoDBMesosframework
AssoonasArangoDBwasdeployedMesoswillkeepyourclusterrunning.ThewebinterfacehasmanymonitoringfacilitiessobesuretomakeyourselffamiliarwiththeDC/OSwebinterface.AsafaulttolerantsystemMesoswilltakecareofmostfailurescenariosautomatically.MesosdoesthatbyrunningArangoDBasasocalled"framework".ThisframeworkhasbeenspecificallybuilttokeepArangoDBrunninginahealthyconditionontheMesoscluster.Fromtimetotimeataskmightfail.TheArangoDBframeworkwillthentakecareofreschedulingthefailedtask.Asitknowsabouttheveryspecificsofeachclustertaskanditsroleitwillautomaticallytakecareofmostfailurescenarios.
Toinspectwhattheframeworkisdoinggotohttp://web-interface-url/mesosinyourbrowser.Locatethetask"arangodb"andinspectstderrinthe"Sandbox".Thiscanbeofinterestforexamplewhenaslavegotlostandtheframeworkisreschedulingthetask.
Cluster:Mesos,DC/OS
299
UsingArangoDB
TouseArangoDBasadatastoreinyourDC/OSclusteryoucanfacilitatetheservicediscoveryofDC/OS.AssumingyoudeployedastandardArangoDBclusterthemesosdnswillknowaboutarangodb.mesos.BydoingaSRVDNSrequest(checkthedocumentationofmesosdns)youcanfindouttheportwheretheinternalHAProxyofArangoDBisrunning.ThiswillofferaroundrobinloadbalancertoaccessallArangoDBcoordinators.
ScalingArangoDB
TochangethesettingsofyourArangoDBClusteraccesstheArangoDBUIandhit"Nodes".Onthescaletabyouwillhavetheabilitytoscaleyourclusterupanddown.
AfterchangingthesettingstheArangoDBframeworkwilltakecareoftherest.ScalingyourclusterupisgenerallyastraightforwardoperationasMesoswillsimplylaunchanothertaskandbedonewithit.Scalingdownisabitmorecomplicatedasthedatafirsthastobemovedtosomeotherplacesothatwillnaturallytakesomewhatlonger.
Pleasenotethatscalingoperationsmightnotalwayswork.ForexampleiftheunderlyingMesosclusteriscompletelysaturatedwithitsrunningtasksscalingupwillnotbepossible.ScalingdownmightalsofailduetotheclusternotbeingabletomoveallshardsofaDBServertoanewdestinationbecauseofsizelimitations.BesuretochecktheoutputoftheArangoDBframework.
Deinstallation
DeinstallingArangoDBisabitmoredifficultasthereismuchstatebeingkeptintheMesosclusterwhichisnotautomaticallycleanedup.Todeinstallfromthecommandlineusethefollowingoneliner:
dcosarangodbuninstall;dcospackageuninstallarangodb
Thiswillfirstcleanupthestateintheclusterandthenuninstallarangodb.
arangodb-cleanup-framework
Shouldyouforgettocleanupthestateyoucandosolaterbyusingthearangodb-cleanup-frameworkcontainer.Otherwiseyoumightnotbeabletodeployanewarangodbinstallation.
ThecleanupframeworkwillannounceitselfasanormalArangoDB.MesoswillrecognizethisandofferallpersistentvolumesitstillhasforArangoDBtothisframework.Thecleanupframeworkwillthenproperlyfreethepersistentvolumes.Finallyitwillcleanupanystateleftinzookeeper(thecentralconfigurationmanagerinaMesoscluster).
Todeploythecleanupframework,followtheinstructionsinthegithubrepository.AfterdeploymentwatchtheoutputinthesandboxoftheMesoswebinterface.Afterawhilethereshouldn'tbeanypersistentresourceoffersanymoreaseverythingwascleanedup.AfterthatyoucandeletethecleanupframeworkagainviaMarathon.
ApacheMesosandMarathon
YoucanalsoinstallArangoDBonabareApacheMesosclusterprovidedthatMarathonisrunningonit.
Doingsohasthefollowingdownsides:
1. ManualMesosclustersetup2. Youneedtoimplementyourownservicediscovery3. Youaremissingthedcoscli4. Installationanddeinstallationaretedious5. YouneedtosetupsomekindofproxytunneltoaccessArangoDBfromtheoutside6. Sparsemonitoringcapabilities
HoweverthesearethingswhichdonotinfluenceArangoDBitselfandoperatingyourclusterlikethisisfullysupported.
InstallingviaMarathon
ToinstallArangoDBviamarathonyouneedaproperconfigfile:
{
Cluster:Mesos,DC/OS
300
"id":"arangodb",
"cpus":0.25,
"mem":256.0,
"ports":[0,0,0],
"instances":1,
"args":[
"framework",
"--framework_name=arangodb",
"--master=zk://172.17.0.2:2181/mesos",
"--zk=zk://172.17.0.2:2181/arangodb",
"--user=",
"--principal=pri",
"--role=arangodb",
"--mode=cluster",
"--async_replication=true",
"--minimal_resources_agent=mem(*):512;cpus(*):0.25;disk(*):512",
"--minimal_resources_dbserver=mem(*):512;cpus(*):0.25;disk(*):1024",
"--minimal_resources_secondary=mem(*):512;cpus(*):0.25;disk(*):1024",
"--minimal_resources_coordinator=mem(*):512;cpus(*):0.25;disk(*):1024",
"--nr_agents=1",
"--nr_dbservers=2",
"--nr_coordinators=2",
"--failover_timeout=86400",
"--arangodb_image=arangodb/arangodb-mesos:3.1",
"--secondaries_with_dbservers=false",
"--coordinators_with_dbservers=false"
],
"container":{
"type":"DOCKER",
"docker":{
"image":"arangodb/arangodb-mesos-framework:3.1",
"network":"HOST"
}
},
"healthChecks":[
{
"protocol":"HTTP",
"path":"/framework/v1/health.json",
"gracePeriodSeconds":3,
"intervalSeconds":10,
"portIndex":0,
"timeoutSeconds":10,
"maxConsecutiveFailures":0
}
]
}
Carefullyreviewthesettings(especiallytheIPsandtheresources).ThenyoucandeploytoMarathon:
curl-XPOST-H"Content-Type:application/json"http://url-of-marathon/v2/[email protected]
AlternativelyusethewebinterfaceofMarathontodeployArangoDB.IthasaJSONmodeandyoucanusetheaboveconfigurationfile.
DeinstallationviaMarathon
AswithDC/OSyoufirstneedtoproperlycleanupanystateleftovers.
TheeasiestistosimplydeleteArangoDBandthendeploythecleanup-framework(seesectionarangodb-cleanup-framework).
Configurationoptions
TheArangodbMesosframeworkhasatonofdifferentoptionswhicharelistedanddescribedhere:https://github.com/arangodb/arangodb-mesos-framework/tree/3.1
Cluster:Mesos,DC/OS
301
AutomaticnativeClusters
SimilarlytohowtheMesosframeworkarangesanArangoDBclusterinaDC/OSenvironmentforyou,arangodbcandothisforyouinaplainenvironment.
Byinvokingthefirstarangodbyoulaunchaprimarynode.Itwillbindanetworkport,andoutputthecommandsyouneedtocut'n'pasteintotheothernodes.Let'sreviewtheprocessofsuchastartuponthreehostsnamedh01,h02,andh03:
arangodb@h01~>arangodb--ownAddressh01:4000
2017/06/1214:59:38Startingarangodbversion0.5.0+git,build5f97368
2017/06/1214:59:38ServingasmasterwithID'52698769'onh01:4000...
2017/06/1214:59:38Waitingfor3serverstoshowup.
2017/06/1214:59:38Usethefollowingcommandstostartotherservers:
arangodb--dataDir=./db2--joinh01:4000
arangodb--dataDir=./db3--joinh01:4000
2017/06/1214:59:38Listeningon0.0.0.0:4000(h01:4000)
Soyoucutthelinesarangodb--data.dir=./db2--starter.join127.0.0.1andexecutethemfortheothernodes.Ifyourunitonanothernodeonyournetwork,replacethe--starter.join127.0.0.1bythepublicIPofthefirsthost.
arangodbh02~>arangodb--dataDir=./db2--joinh01:4000
2017/06/1214:48:50Startingarangodbversion0.5.0+git,build5f97368
2017/06/1214:48:50Contactingmasterh01:4000...
2017/06/1214:48:50Waitingfor3serverstoshowup...
2017/06/1214:48:50Listeningon0.0.0.0:4000(:4000)
arangodbh03~>arangodb--dataDir=./db3--joinh01:4000
2017/06/1214:48:50Startingarangodbversion0.5.0+git,build5f97368
2017/06/1214:48:50Contactingmasterh01:4000...
2017/06/1214:48:50Waitingfor3serverstoshowup...
2017/06/1214:48:50Listeningon0.0.0.0:4000(:4000)
Oncethetwootherprocessesjoinedthecluster,andstartedtheirArangoDBserverprocesses(thismaytakeawhiledependingonyoursystem),itwillinformyouwheretoconnecttheClusterfromaBrowser,shelloryourprogramm:
...
2017/06/1214:55:21coordinatorupandrunning.
Atthispointyoumayaccessyourclusterateithercoordinatorendpoint,http://h01:4002/,http://h02:4002/orhttp://h03:4002/.
AutomaticnativelocaltestClusters
Ifyouonlywantalocaltestcluster,youcanrunasinglestarterwiththe--starter.localargument.Itwillstarta3"machine"clusteronyourlocalPC.
arangodb--starter.local
Note.AlocalclusterisintendedonlyfortestpurposessinceafailureofasinglePCwillbringdowntheentirecluster.
AutomaticDockerClusters
ArangoDBStartercanalsobeusedtolaunchclustersbasedondockercontainers.Itsabitmorecomplicated,sinceyouneedtoprovideinformationaboutyourenvironmentthatcan'tbeautodetected.
IntheDockerworldyouneedtotakecareaboutwherepersistantdataisstored,sincecontainersareintendedtobevolatile.Weuseavolumenamedarangodb1here:
Cluster:Generic&Docker
302
dockervolumecreatearangodb1
(Youcanuseanytypeofdockervolumethatfitsyoursetupinstead.)
WethenneedtodeterminethetheIPofthedockerhostwhereyouintendtorunArangoDBstarteron.Dependingonyouroperatingsystemexecuteipaddr,ifconfigoripconfigtodetermineyourlocalipaddress.
192.168.1.0/24deveth0protokernelscopelinksrc192.168.1.32
SothisexampleusestheIP192.168.1.32:
dockerrun-it--name=adb1--rm-p8528:8528\
-varangodb1:/data\
-v/var/run/docker.sock:/var/run/docker.sock\
arangodb/arangodb-starter\
--starter.address=192.168.1.32
Itwillstartthemasterinstance,andcommandyoutostarttheslaveinstances:
Unabletofindimage'arangodb/arangodb-starter:latest'locally
latest:Pullingfromarangodb/arangodb-starter
Digest:sha256:b87d20c0b4757b7daa4cb7a9f55cb130c90a09ddfd0366a91970bcf31a7fd5a4
Status:Downloadednewerimageforarangodb/arangodb-starter:latest
2017/06/1213:26:14Startingarangodbversion0.7.1,buildf128884
2017/06/1213:26:14ServingasmasterwithID'46a2b40d'on192.168.1.32:8528...
2017/06/1213:26:14Waitingfor3serverstoshowup.
2017/06/1213:26:14Usethefollowingcommandstostartotherservers:
dockervolumecreatearangodb2&&\
dockerrun-it--name=adb2--rm-p8533:8528-varangodb2:/data\
-v/var/run/docker.sock:/var/run/docker.sockarangodb/arangodb-starter:0.7\
--starter.address=192.168.1.32--starter.join=192.168.1.32
dockervolumecreatearangodb3&&\
dockerrun-it--name=adb3--rm-p8538:8528-varangodb3:/data\
-v/var/run/docker.sock:/var/run/docker.sockarangodb/arangodb-starter:0.7\
--starter.address=192.168.1.32--starter.join=192.168.1.32
Onceyoustarttheotherinstances,itwillcontinuelikethis:
2017/05/1109:05:45Addedmaster'fc673b3b':192.168.1.32,portOffset:0
2017/05/1109:05:45Addednewpeer'e98ea757':192.168.1.32,portOffset:5
2017/05/1109:05:50Addednewpeer'eb01d0ef':192.168.1.32,portOffset:10
2017/05/1109:05:51Startingservice...
2017/05/1109:05:51Lookingforarunninginstanceofagentonport8531
2017/05/1109:05:51Startingagentonport8531
2017/05/1109:05:52Lookingforarunninginstanceofdbserveronport8530
2017/05/1109:05:52Startingdbserveronport8530
2017/05/1109:05:53Lookingforarunninginstanceofcoordinatoronport8529
2017/05/1109:05:53Startingcoordinatoronport8529
2017/05/1109:05:58agentupandrunning(version3.2.devel).
2017/05/1109:06:15dbserverupandrunning(version3.2.devel).
2017/05/1109:06:31coordinatorupandrunning(version3.2.devel).
Andatleastittellsyouwhereyoucanworkwithyourcluster:
2017/05/1109:06:31Yourclustercannowbeaccessedwithabrowserat`http://192.168.1.32:8529`or
2017/05/1109:06:31using`arangosh--server.endpointtcp://192.168.1.32:8529`.
Underthehood
Thefirstarangodbyouran(asshownabove)willbecomethemasterinyoursetup,the--starter.joinwillbetheslaves.
Cluster:Generic&Docker
303
ThemasterdetermineswhichArangoDBserverprocessestolaunchonwhichslave,andhowtheyshouldcommunicate.Itwillthenlaunchtheserverprocessesandmonitorthem.Onceithasdetectedthatthesetupiscompleteyouwillgettheprompt.Themasterwillsavethesetupforsubsequentstarts.
MorecomplicatedsetupoptionscanbefoundinArangoDBStartersReadme.
Cluster:Generic&Docker
304
Datacentertodatacenterreplication.
About
Atsomepointinthegrowsofadatabase,therecomesaneedforreplicatingitacrossmultipledatacenters.
Reasonsforthatcanbe:
Fallbackincaseofadisasterinonedatacenter.RegionalavailabilitySeparationofconcerns
Andmanymore.
ThistutorialdescribeswhattheArangoSyncdatacentertodatacenterreplicationsolution(ArangoSyncfromnowon)offers,whentouseit,whennottouseitandhowtoconfigure,operate,troubleshootit&keepitsafe.
Whatisit
ArangoSyncisasolutionthatenablesyoutoasynchronouslyreplicatetheentirestructureandcontentinanArangoDBclusterinoneplacetoaclusterinanotherplace.Typicallyitisusedfromonedatacentertoanother.Itisnotasolutionforreplicatingsingleserverinstances.
ThereplicationdonebyArangoSyncinasynchronous.Thatmeansthatwhenaclientiswritingdataintothesourcedatacenter,itwillconsidertherequestfinishedbeforethedatahasbeenreplicatedtotheotherdatacenter.Thetimeneededtocompletelyreplicatechangestotheotherdatacenteristypicallyintheorderofseconds,butthiscanvarysignificantlydependingonload,network&computercapacity.
ArangoSyncperformsreplicationinasingledirectiononly.ThatmeansthatyoucanreplicatedatafromclusterAtoclusterBorfromclusterBtoclusterA,butneveratthesametime.Datamodifiedinthedestinationclusterwillbelost!
Replicationisacompletelyautonomousprocess.Onceitisconfigureditisdesignedtorun24/7withoutfrequentmanualintervention.Thisdoesnotmeanthatitrequiresnomaintenanceorattentionatall.Aswithanydistributedsystemsomeattentionisneededtomonitoritsoperationandkeepitsecure(e.g.certificate&passwordrotation).
Onceconfigured,ArangoSyncwillreplicatebothstructureanddataofanentirecluster.Thismeansthatthereisnoneedtomakeadditionalconfigurationchangeswhenadding/removingdatabasesorcollections.Alsometadatasuchasusers,foxxapplication&jobsareautomaticallyreplicated.
Whentouseit...andwhennot
ArangoSyncisagoodsolutioninallcaseswhereyouwanttoreplicatedatafromoneclustertoanotherwithouttherequirementthatthedataisavailableimmediatelyintheothercluster.
ArangoSyncisnotagoodsolutionwhenoneofthefollowingapplies:
YouwanttoreplicatedatafromclusterAtoclusterBandfromclusterBtoclusterAatthesametime.Youneedsynchronousreplicationbetween2clusters.ThereisnonetworkconnectionbetweeclusterAandB.Youwantcompletecontroloverwhichdatabase,collection&documentsarereplicateandwhichnot.
Requirements
TouseArangoSyncyouneedthefollowing:
Twodatacenters,eachrunninganArangoDBEnterprisecluster,version3.3orhigher.
MultipleDatacenters
305
Anetworkconnectionbetweenbothdatacenterswithaccessibleendpointsforseveralcomponents(seeindividualcomponentsfordetails).TLScertificatesforArangoSyncmasterinstances(canbeself-signed).TLScertificatesforKafkabrokers(canbeself-signed).Optional(butrecommended)TLScertificatesforArangoDBclusters(canbeself-signed).ClientcertificatesCAforArangoSyncmasters(typicallyself-signed).ClientcertificatesforArangoSyncmasters(typicallyself-signed).Atleast2instancesoftheArangoSyncmasterineachdatacenter.OneinstancesoftheArangoSyncworkeroneverymachineineachdatacenter.
Note:Inseveralplacesyouwillneeda(x509)certificate.Thecertificatessectionbelowprovidesmoreguidanceforcreatingandrenewingthesecertificates.
Besidestheabovelist,youprobablywanttousethefollowing:
Anorchestratortokeepallcomponentsrunning.Inthistutorialwewillusesystemdasanexample.Alogfilecollectorforcentralizedcollection&accesstothelogsofallcomponents.Ametricscollector&viewingsolutionsuchasPrometheus+Grafana.
Deployment
Inthefollowingparagraphsyou'lllearnhowtodeployallthecomponentsneededfordatacentertodatacenterreplication.
ArangoDBcluster
ThereareseveralwaystostartanArangoDBcluster.InthistutorialwewillfocusonourrecommendedwaytostartArangoDB:theArangoDBstarter.
Datacentertodatacenterreplicationrequirestherocksdbstorageengine.Inthistutorialtheexamplesetupwillhaverocksdbenabled.Ifyouchoosetodeploywithadifferentstrategykeepinmindtosetthestorageengine.
FortheotherpossibilitiestodeployanArangoDBclusterpleaserefertotheDeploymentchapter:
TODOONCEWEMERGEWITHTHESTANDARDDOCUMENTATION:INLINELINK
Thestartersimplifiesthingsfortheoperatorandwillcoordinateadistributedclusterstartupacrossseveralmachinesandassignclusterrolesautomatically.
Whenstartedonseveralmachinesandenoughmachineshavejoined,thestarterswillstartagents,coordinatorsanddbserversonthesemachines.
Whenrunningthestarterwillsuperviseitschildtasks(namelycoordinators,dbserversandagents)andrestartthemincaseoffailure.
Tostarttheclusterusingasystemdunitfileusethefollowing:
[Unit]
Description=RuntheArangoDBStarter
After=network.target
[Service]
Restart=on-failure
EnvironmentFile=/etc/arangodb.env
EnvironmentFile=/etc/arangodb.env.local
Environment=DATADIR=/var/lib/arangodb/cluster
ExecStartPre=/usr/bin/sh-c"mkdir-p${DATADIR}"
ExecStart=/usr/bin/arangodb\
--starter.address=${PRIVATEIP}\
--starter.data-dir=${DATADIR}\
--starter.join=${STARTERENDPOINTS}\
--server.storage-engine=rocksdb\
--auth.jwt-secret=${CLUSTERSECRETPATH}
TimeoutStopSec=60
[Install]
WantedBy=multi-user.target
MultipleDatacenters
306
Notethatwesetrocksdbintheunitservicefile.
Clusterauthentication
Thecommunicationbetweentheclusternodesuseatoken(JWT)toauthenticate.Thismustbesharedbetweenclusternodes.
Sharingsecretsisobviouslyaverydelicatetopic.Theaboveworkflowassumesthattheoperatorwillputasecretinafilenamed${CLUSTERSECRETPATH}.
WerecommendtouseadedicatedsystemformanagingsecretslikeHashiCorps'VaultorthesecretmanagementofDC/OS.
Requiredports
Assoonasenoughmachineshavejoined,thestarterwillbeginstartingagents,coordinatorsanddbservers.
Eachofthesetasksneedsaporttocommunicate.Pleasemakesurethatthefollowingportsareavailableonallmachines:
8529forcoordinators8530fordbservers8531foragents
Thestarteritselfwilluseport8528.
Kafka&Zookeeper
HowtodeployzookeeperHowtodeploykafkaAccessibleports
SyncMaster
TheSyncMasterisresponsibleformanagingallsynchronization,creatingtasksandassigningthosetoworkers.Atleast2instancesmutsbedeployedineachdatacenter.Oneinstancewillbethe"leader",theotherwillbeaninactiveslave.Whentheleaderisgoneforashortwhile,oneoftheotherinstanceswilltakeover.
Withclustersofasignificantsize,thesyncmasterwillrequireasignificantsetofresources.Thereforeitisrecommendedtodeploysyncmastersontheirownservers,equipedwithsufficientCPUpowerandmemorycapacity.
Tostartasyncmasterusingasystemdservice,useaunitlikethis:
[Unit]
Description=RunArangoSyncinmastermode
After=network.target
[Service]
Restart=on-failure
EnvironmentFile=/etc/arangodb.env
EnvironmentFile=/etc/arangodb.env.local
ExecStart=/usr/sbin/arangosyncrunmaster\
--log.level=debug\
--cluster.endpoint=${CLUSTERENDPOINTS}\
--cluster.jwtSecret=${CLUSTERSECRET}\
--server.keyfile=${CERTIFICATEDIR}/tls.keyfile\
--server.client-cafile=${CERTIFICATEDIR}/client-auth-ca.crt\
--server.endpoint=https://${PUBLICIP}:${MASTERPORT}\
--server.port=${MASTERPORT}\
--master.jwtSecret=${MASTERSECRET}\
--mq.type=kafka\
--mq.kafka-addr=${KAFKAENDPOINTS}\
--mq.kafka-client-keyfile=${CERTIFICATEDIR}/kafka-client.key\
--mq.kafka-cacert=${CERTIFICATEDIR}/tls-ca.crt\
TimeoutStopSec=60
[Install]
WantedBy=multi-user.target
MultipleDatacenters
307
ThesyncmasterneedsaTLSservercertificateandaIfyouwanttheservicetocreateaTLScertificate&clientauthenticationcertificate,forauthenticatingwithsyncmastersinanotherdatacenter,foreverystart,addthistotheServicesection.
ExecStartPre=/usr/bin/sh-c"mkdir-p${CERTIFICATEDIR}"
ExecStartPre=/usr/sbin/arangosynccreatetlskeyfile\
--cacert=${CERTIFICATEDIR}/tls-ca.crt\
--cakey=${CERTIFICATEDIR}/tls-ca.key\
--keyfile=${CERTIFICATEDIR}/tls.keyfile\
--host=${PUBLICIP}\
--host=${PRIVATEIP}\
--host=${HOST}
ExecStartPre=/usr/sbin/arangosynccreateclient-authkeyfile\
--cacert=${CERTIFICATEDIR}/tls-ca.crt\
--cakey=${CERTIFICATEDIR}/tls-ca.key\
--keyfile=${CERTIFICATEDIR}/kafka-client.key\
--host=${PUBLICIP}\
--host=${PRIVATEIP}\
--host=${HOST}
ThesyncmastermustbereachableonaTCPport${MASTERPORT}(usedwith--server.portoption).Thisportmustbereachablefrominsidethedatacenter(bysyncworkersandoperations)andfrominsideoftheotherdatacenter(bysyncmastersintheotherdatacenter).
SyncWorkers
TheSyncWorkerisresponsibleforexecutingsynchronizationtasks.Foroptimalperformanceatleast1workerinstancemustbeplacedoneverymachinethathasanArangoDBdbserverrunning.Thisensuresthattaskscanbeexecutedwithminimalnetworktrafficoutsideofthemachine.
SincesyncworkerswillautomaticallystoponcetheirTLSservercertificateexpires(whichissetto2yearsbydefault),itisrecommendedtorunatleast2instancesofaworkeroneverymachineinthedatacenter.Thatway,taskscanstillbeassignedinthemostoptimalway,evenwhenaworkerintemporarilydownforarestart.
Tostartasyncworkerusingasystemdservice,useaunitlikethis:
[Unit]
Description=RunArangoSyncinworkermode
After=network.target
[Service]
Restart=on-failure
EnvironmentFile=/etc/arangodb.env
EnvironmentFile=/etc/arangodb.env.local
Environment=PORT=8729
ExecStart=/usr/sbin/arangosyncrunworker\
--log.level=debug\
--server.port=${PORT}\
--server.endpoint=https://${PRIVATEIP}:${PORT}\
--master.endpoint=${MASTERENDPOINTS}\
--master.jwtSecret=${MASTERSECRET}
TimeoutStopSec=60
[Install]
WantedBy=multi-user.target
ThesyncworkermustbereachableonaTCPport${PORT}(usedwith--server.portoption).Thisportmustbereachablefrominsidethedatacenter(bysyncmasters).
Prometheus&Grafana(optional)
ArangoSyncprovidesmetricsinaformatsupportedbyPrometheus.WealsoprovideastandardsetofdashboardsforviewingthosemetricsinGrafana.
Ifyouwanttousethesetools,gototheirwebsitesforinstructionsonhowtodeploythem.
Afterdeployment,youmustconfigureprometheususingaconfigurationfilethatinstructsitaboutwhichtargetstoscrape.ForArangoSyncyoushouldconfigurescrapetargetsforallsyncmastersandallsyncworkers.Todoso,youcanuseaconfigurationsuchasthis:
MultipleDatacenters
308
global:
scrape_interval:10s#scrapetargetsevery10seconds.
scrape_configs:
#Scrapsyncmasters
-job_name:'sync_master'
scheme:'https'
bearer_token:"${MONITORINGTOKEN}"
tls_config:
insecure_skip_verify:true
static_configs:
-targets:
-"${IPMASTERA1}:8629"
-"${IPMASTERA2}:8629"
-"${IPMASTERB1}:8629"
-"${IPMASTERB2}:8629"
labels:
type:"master"
relabel_configs:
-source_labels:[__address__]
regex:${IPMASTERA1}\:8629|${IPMASTERA2}\:8629
target_label:dc
replacement:A
-source_labels:[__address__]
regex:${IPMASTERB1}\:8629|${IPMASTERB2}\:8629
target_label:dc
replacement:B
-source_labels:[__address__]
regex:${IPMASTERA1}\:8629|${IPMASTERB1}\:8629
target_label:instance
replacement:1
-source_labels:[__address__]
regex:${IPMASTERA2}\:8629|${IPMASTERB2}\:8629
target_label:instance
replacement:2
#Scrapsyncworkers
-job_name:'sync_worker'
scheme:'https'
bearer_token:"${MONITORINGTOKEN}"
tls_config:
insecure_skip_verify:true
static_configs:
-targets:
-"${IPWORKERA1}:8729"
-"${IPWORKERA2}:8729"
-"${IPWORKERB1}:8729"
-"${IPWORKERB2}:8729"
labels:
type:"worker"
relabel_configs:
-source_labels:[__address__]
regex:${IPWORKERA1}\:8729|${IPWORKERA2}\:8729
target_label:dc
replacement:A
-source_labels:[__address__]
regex:${IPWORKERB1}\:8729|${IPWORKERB2}\:8729
target_label:dc
replacement:B
-source_labels:[__address__]
regex:${IPWORKERA1}\:8729|${IPWORKERB1}\:8729
target_label:instance
replacement:1
-source_labels:[__address__]
regex:${IPWORKERA2}\:8729|${IPWORKERB2}\:8729
target_label:instance
replacement:2
Note:Theaboveexampleassumes2datacenters,with2syncmasters&2syncworkersperdatacenter.Youhavetoreplaceall${...}variablesintheaboveconfigurationwithapplicablevaluesfromyourenvironment.
Configuration
MultipleDatacenters
309
OnceallcomponentsoftheArangoSyncsolutionhavebeendeployedandarerunningproperly,ArangoSyncwillnotautomaticallyreplicatedatabasestructureandcontent.Forthat,itisisneededtoconfiguresynchronization.
Toconfiguresynchronization,youneedthefollowing:
Theendpointofthesyncmasterinthetargetdatacenter.Theendpointofthesyncmasterinthesourcedatacenter.Acertificate(inkeyfileformat)usedforclientauthenticationofthesyncmaster(withthesyncmasterinthesourcedatacenter).ACAcertificate(publickeyonly)forverifyingtheintegrityofthesyncmasters.Ausername+passwordpair(orclientcertificate)forauthenticatingtheconfigurerequirewiththesyncmaster(inthetargetdatacenter)
Withthatinformation,run:
arangosyncconfiguresync\
--master.endpoint=<endpointsofsyncmastersintargetdatacenter>\
--master.keyfile=<keyfileofofsyncmastersintargetdatacenter>\
--source.endpoint=<endpointsofsyncmastersinsourcedatacenter>\
--source.cacert=<publickeyofCAcertificateusedtoverifysyncmasterinsourcedatacenter>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>
Thecommandwillfinishquickly.Afterwardsitwilltakesometimeuntiltheclustersinbothdatacentersareinsync.
Usethefollowingcommandtoinspectthestatusofthesynchronizationofadatacenter:
arangosyncgetstatus\
--master.endpoint=<endpointsofsyncmastersindatacenterofinterest>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>\
-v
Note:Invokingthiscommandonthetargetdatacenterwillreturndifferentresultsfrominvokingitonthesourcedatacenter.Youneedinsightinbothresultstogeta"completepicture".
Wherethegetstatuscommandgivesinsightinthestatusofsynchronization,therearemoredetailedcommandstogiveinsightintasks®isteredworkers.
Usethefollowingcommandtogetalistofallsynchronizationtasksinadatacenter:
arangosyncgettasks\
--master.endpoint=<endpointsofsyncmastersindatacenterofinterest>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>\
-v
Usethefollowingcommandtogetalistofallmastersinadatacenterandknowwhichmasteristhecurrentleader:
arangosyncgetmasters\
--master.endpoint=<endpointsofsyncmastersindatacenterofinterest>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>\
-v
Usethefollowingcommandtogetalistofallworkersinadatacenter:
arangosyncgetworkers\
--master.endpoint=<endpointsofsyncmastersindatacenterofinterest>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>\
-v
Stopsynchronization
MultipleDatacenters
310
Ifyounolongerwanttosynchronizedatafromasourcetoatargetdatacenteryoumuststopit.Todoso,runthefollowingcommand:
arangosyncstopsync\
--master.endpoint=<endpointsofsyncmastersintargetdatacenter>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>
Thecommandwillwaituntilsynchronizationhascompletelystoppedbeforereturning.Ifthesynchronizationisnotcompletelystoppedwithinareasonableperiod(2minutesbydefault)thecommandwillfail.
Ifthesourcedatacenterisnolongeravailableitisnotpossibletostopsynchronizationinagracefulmanner.Ifthathappensabortthesynchronizationwiththefollowingcommand:
arangosyncabortsync\
--master.endpoint=<endpointsofsyncmastersintargetdatacenter>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>
Ifthesourcedatacenterrecoversafteranabortsynchasbeenexecuted,itisneededto"cleanup"ArangoSyncinthesourcedatacenter.Todoso,executethefollowingcommand:
arangosyncabortoutgoingsync\
--master.endpoint=<endpointsofsyncmastersinsourcedatacenter>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>
Reversingsynchronizationdirection
Ifyouwanttoreversethedirectionofsynchronization(e.g.afterafailureindatacenterAandyouswitchedtothedatacenterBforfallback),youmustfirststop(orabort)theoriginalsynchronization.
Oncethatisfinished(andcleanuphasbeenappliedincaseofabort),youmustnowconfigurethesynchronizationagain,butwithswappedsource&targetsettings.
Operations&Maintenance
ArangoSyncisadistributedsystemwithalotdifferentcomponents.Aswithanysuchsystem,itrequiressome,butnotalot,ofoperationalsupport.
Whatmeansareavailabletomonitorstatus
AllofthecomponentsofArangoSyncprovidemeanstomonitortheirstatus.Belowyou'llfindanoverviewpercomponent.
Syncmaster&workers:Thearangosyncserversrunningaseithermasterorworker,provide:
AstatusAPI,seearangosyncgetstatus.Makesurethatallstatusesreportrunning.Forevenmoredetailthefollowingcommandsarealsoavailable:arangosyncgettasks,arangosyncgetmasters&arangosyncgetworkers.Alogonthestandardoutput.Loglevelscanbeconfiguredusing--log.levelsettings.AmetricsAPIGET/metrics.ThisAPIiscompatiblewithPrometheus.SampleGrafanadashboardsforinspectingthesemetricsareavailable.
ArangoDBcluster:ThearangodserversthatmakeuptheArangoDBclusterprovide:
Alogfile.Thisisconfigurablewithsettingswithalog.prefix.E.g.--log.output=file://myLogFileor--log.level=info.AstatisticsAPIGET/_admin/statistics
Kafkacluster:Thekafkabrokersprovide:
Alogfile,seesettingswithlog.prefixinitsserver.propertiesconfigurationfile.Zookeeper:Thezookeeperagentsprovide:
Alogonstandardoutput.
MultipleDatacenters
311
Whattolookforwhilemonitoringstatus
TheveryfirstthingtodowhenmonitoringthestatusofArangoSyncistolookintothestatusprovidedbyarangosyncgetstatus...-v.Whennoteverythingisintherunningstate(onbothdatacenters),thisisanindicationthatsomethingmaybewrong.Incasethathappens,giveitsometime(incrementalsynchronizationmaytakequitesometimeforlargecollections)andlookatthestatusagain.Ifthestatusesdonotchange(orchange,butnotreachrunning)itistimetoinspectsthemetrics&logfiles.Whenthemetricsorlogsseemtoindicateaprobleminasyncmasterorworker,itissafetorestartit,aslongasonly1instanceisrestartedatatime.Giverestartedinstancessometimeto"catchup".
Whattodowhenproblemsremain
Whenaproblemremainsandrestartingmasters/workersdoesnotsolvetheproblem,contactsupport.Makesuretoincludeprovidesupportwiththefollowinginformation:
Outputofarangosyncgetversion...onbothdatacenters.Outputofarangosyncgetstatus...-vonbothdatacenters.Outputofarangosyncgettasks...-vonbothdatacenters.Outputofarangosyncgetmasters...-vonbothdatacenters.Outputofarangosyncgetworkers...-vonbothdatacenters.LogfilesofallcomponentsAcompletedescriptionoftheproblemyouobservedandwhatyoudidtoresolveit.
HowtomonitorstatusofArangoSync
HowtokeepitaliveWhattodoincaseoffailuresorbugs
Whattodowhenasourcedatacenterisdown
WhenyouuseArangoSyncforbackupofyourclusterfromonedatacentertoanotherandthesourcedatacenterhasacompleteoutage,youmayconsiderswitchingyourapplicationstothetarget(backup)datacenter.
Thisiswhatyoumustdointhatcase.
1. Stopconfigurationusingarangosyncstopsync....Whenthesourcedatacenteriscompletelyunresponsivethiswillnotsucceed.Inthatcaseusearangosyncabortsync....SeeConfigurationforhowtocleanupthesourcedatacenterwhenitbecomesavailableagain.
2. Verifythatconfigurationhascompletelystoppedusingarangosyncgetstatus...-v.3. Reconfigureyourapplicationstousethetarget(backup)datacenter.
Whentheoriginalsourcedatacenterisrestored,youmayswitchrolesandmakeitthetargetdatacenter.Todoso,usearangosyncconfiguresync...asdescribedinConfiguration.
Whattodoincaseofaplannednetworkoutage.
AllArangoSynctaskssendoutheartbeatmessagesouttotheotherdatacentertoindicate"itisstillalive".Theotherdatacenterassumestheconnectionis"outofsync"whenitdoesnotreceiveanymessagesforacertainperiodoftime.
Ifyou'replanningsomesortofmaintenancewhereyouknowtheconnectivitywillbelostforsometime(e.g.3hours),youcanprepareArangoSyncforthatsuchthatitwillholdofre-synchronizationforagivenperiodoftime.
Todoso,onbothdatacenters,run:
arangosyncsetmessagetimeout\
--master.endpoint=<endpointsofsyncmastersinthedatacenter>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>\
3h
ThelastargumentistheperiodthatArangoSyncshouldhold-ofresynchronizationfor.Thiscanbeminutes(e.g.15m)orhours(e.g.3h).
MultipleDatacenters
312
Ifmaintenanceistakinglongerthanexpected,youcanusethesamecommandtheextendtheholdofperiod(e.g.to4h).
Afterthemaintenance,usethesamecommandrestoretheholdofperiodtoitsdefaultof1h.
Whattodoincaseofadocumentthatexceedsthemessagequeuelimits.
Ifyouinsert/updateadocumentinacollectionandthesizeofthatdocumentislargerthanthemaximummessagesizeofyourmessagequeue,thecollectionwillnolongerbeabletosynchronize.Itwillgointoafailedstate.
Torecoverfromthat,firstremovethedocumentfromtheArangoDBclusterinthesourcedatacenter.Afterthat,foreachfailedshard,run:
arangosyncresetfailedshard\
--master.endpoint=<endpointsofsyncmastersinthedatacenter>\
--auth.user=<usernameusedforauthenticationofthiscommand>\
--auth.password=<passwordofauth.user>\
--database=<nameofthedatabase>\
--collection=<nameofthecollection>\
--shard=<indexoftheshard(startingat0)>
Afterthiscommand,anewsetoftaskswillbestartedtosynchronizetheshard.Itcantakesometimefortheshardtoreachrunningstate.
Metrics
ArangoSync(master&worker)providemetricsthatcanbeusedformonitoringtheArangoSyncsolution.ThesemetricsareavailableusingthefollowingHTTPSendpoints:
GET/metrics:ProvidesmetricsinaformatsupportedbyPrometheus.GET/metrics.json:ProvidesthesamemetricsinJSONformat.
Bothendpointsincludehelpinformationpermetrics.
Note:Bothendpointsrequireauthentication.Besidestheusualauthenticationmethodstheseendpointsarealsoaccessibleusingaspecialbearertokenspecifiedusingthe--monitoring.tokencommandlineoption.
ThePrometheusoutput(/metrics)lookslikethis:
...
#HELParangosync_master_worker_registrationsTotalnumberofregistrations
#TYPEarangosync_master_worker_registrationscounter
arangosync_master_worker_registrations2
#HELParangosync_master_worker_storageNumberoftimesworkerinfoisstored,loaded
#TYPEarangosync_master_worker_storagecounter
arangosync_master_worker_storage{kind="",op="save",result="success"}20
arangosync_master_worker_storage{kind="empty",op="load",result="success"}1
...
TheJSONoutput(/metrics.json)lookslikethis:
{
...
"arangosync_master_worker_registrations":{
"help":"Totalnumberofregistrations",
"type":"counter",
"samples":[
{
"value":2
}
]
},
"arangosync_master_worker_storage":{
"help":"Numberoftimesworkerinfoisstored,loaded",
"type":"counter",
"samples":[
{
"value":8,
MultipleDatacenters
313
"labels":{
"kind":"",
"op":"save",
"result":"success"
}
},
{
"value":1,
"labels":{
"kind":"empty",
"op":"load",
"result":"success"
}
}
]
}
...
}
Hint:Togetalistofametricsandtheirhelpinformation,run:
aliasjq='dockerrun--rm-irealguess/jqjq'
curl-sk-u"<user>:<password>"https://<syncmaster-IP>:8629/metrics.json|\
jq'with_entries({key:.key,value:.value.help})'
Security
Firewallsettings
ThecomponentsofArangoSyncuse(TCP)networkconnectionstocommunicatewitheachother.Belowyou'llfindanoverviewoftheseconnectionsandtheTCPportsthatshouldbeaccessible.
1. Thesyncmastersmustbeallowedtoconnecttothefollowingcomponentswithinthesamedatacenter:
ArangoDBagentsandcoordinators(defaultports:8531and8529)Kafkabrokers(defaultport9092)Syncworkers(defaultport8729)
Additionallythesyncmastersmustbeallowedtoconnecttothesyncmastersintheotherdatacenter.
Bydefaultthesyncmasterswilloperateonport8629.
2. Thesyncworkersmustbeallowedtoconnecttothefollowingcomponentswithinthesamedatacenter:
ArangoDBcoordinators(defaultport8529)Kafkabrokers(defaultport9092)Syncmasters(defaultport8629)
Bydefaultthesyncworkerswilloperateonport8729.
AdditionallythesyncworkersmustbeallowedtoconnecttotheKafkabrokersintheotherdatacenter.
3. Kafka
Thekafkabrokersmustbeallowedtoconnecttothefollowingcomponentswithinthesamedatacenter:
Otherkafkabrokers(defaultport9092)Zookeeper(defaultports2181,2888and3888)
Thedefaultportforkafkais9092.Thedefaultkafkainstallationwillalsoexposesomeprometheusmetricsonport7071.Togainmoreinsightintokafkaopenthisportforyourprometheusinstallation.
4. Zookeeper
Thezookeeperagentsmustbeallowedtoconnecttothefollowingcomponentswithinthesamedatacenter:
OtherzookeeperagentsThesetuphereisabitspecialaszookeeperuses3portsfordifferentoperations.Allagentsneedtobeabletoconnecttoalloftheseports.
MultipleDatacenters
314
BydefaultZookeeperuses:
port2181forclientcommunicationport2888forfollowercommunicationport3888forleaderelections
Certificates
DigitalcertificatesareusedinmanyplacesinArangoSyncforbothencryptionandauthentication.
InArangoSyncallnetworkconnectionsareusingTransportLayerSecurity(TLS),asetofprotocolsthatensurethatallnetworktrafficisencrypted.ForthisTLScertificatesareused.TheserversideofthenetworkconnectionoffersaTLScertificate.Thiscertificateis(often)verifiedbytheclientsideofthenetworkconnection,toensurethatthecertificateissignedbyatrustedCertificateAuthority(CA).Thisensurestheintegrityoftheserver.Inseveralplacesadditionalcertificatesareusedforauthentication.Inthosecasestheclientsideoftheconnectionoffersaclientcertificate(ontopofanexistingTLSconnection).Theserversideoftheconnectionusestheclientcertificatetoauthenticatetheclientand(optionally)decideswhichrightsshouldbeassignedtotheclient.
Note:ArangoSyncdoesallowtheuseofcertificatessignedbyawellknowCA(eg.verisign)howeveritismoreconvenient(andcommon)touseyourownCA.
Formats
Allcertificatesarex509certificateswithapublickey,aprivatekeyandanoptionalchainofcertificatesusedtosignthecertificate(thischainistypicallyprovidedbytheCertificateAuthority(CA)).Dependingontheiruse,certificatesstoredinadifferentformat.
Thefollowingformatsareused:
Publickeyonly(.crt):Afilethatcontainsonlythepublickeyofacertificatewithanoptionalchainofparentcertificates(publickeysofcertificatesusedtosignedthecertificate).Sincethisformatcontainsonlypublickeys,itisnotaproblemifitscontentsareexposed.Itmuststillbestoreitinasafeplacetoavoidlosingit.Privatekeyonly(.key):Afilethatcontainsonlytheprivatekeyofacertificate.Itisvitaltoprotectthesefilesandstoretheminasafeplace.Keyfilewithpublic&privatekey(.keyfile):Afilethatcontainsthepublickeyofacertificate,anoptionalchainofparentcertificatesandaprivatekey.Sincethisformatalsocontainsaprivatekey,itisvitaltoprotectthesefilesandstoretheminasafeplace.Javakeystore(.jks):Afilecontainingasetofpublicandprivatekeys.Itispossibletoprotectaccesstothecontentofthisfileusingakeystorepassword.Sincethisformatcancontainprivatekeys,itisvitaltoprotectthesefilesandstoretheminasafeplace(evenwhenitscontentisprotectedwithakeystorepassword).
Creatingcertificates
ArangoSyncprovidescommandstocreateallcertificatesneeded.
TLSservercertificates
TocreateacertificateusedforTLSserversinthekeyfileformat,youneedthepublickeyoftheCA(--cacert),theprivatekeyoftheCA(--cakey)andoneormorehostnames(orIPaddresses).Thenrun:
arangosynccreatetlskeyfile\
--cacert=my-tls-ca.crt--cakey=my-tls-ca.key\
--host=<hostname>\
--keyfile=my-tls-cert.keyfile
Makesuretostorethegeneratedkeyfile(my-tls-cert.keyfile)inasafeplace.
MultipleDatacenters
315
TocreateacertificateusedforTLSserversinthecrt&keyformat,youneedthepublickeyoftheCA(--cacert),theprivatekeyoftheCA(--cakey)andoneormorehostnames(orIPaddresses).Thenrun:
arangosynccreatetlscertificate\
--cacert=my-tls-ca.crt--cakey=my-tls-ca.key\
--host=<hostname>\
--cert=my-tls-cert.crt\
--key=my-tls-cert.key\
Makesuretoprotectandstorethegeneratedfiles(my-tls-cert.crt&my-tls-cert.key)inasafeplace.
Clientauthenticationcertificates
Tocreateacertificateusedforclientauthenticationinthekeyfileformat,youneedthepublickeyoftheCA(--cacert),theprivatekeyoftheCA(--cakey)andoneormorehostnames(orIPaddresses)oremailaddresses.Thenrun:
arangosynccreateclient-authkeyfile\
--cacert=my-client-auth-ca.crt--cakey=my-client-auth-ca.key\
[--host=<hostname>|--email=<emailaddress>]\
--keyfile=my-client-auth-cert.keyfile
Makesuretoprotectandstorethegeneratedkeyfile(my-client-auth-cert.keyfile)inasafeplace.
CAcertificates
TocreateaCAcertificateusedtosignTLScertificates,run:
arangosynccreatetlsca\
--cert=my-tls-ca.crt--key=my-tls-ca.key
Makesuretoprotectandstorebothgeneratedfiles(my-tls-ca.crt&my-tls-ca.key)inasafeplace.Note:CAcertificateshaveamuchlongerlifetimethannormalcertificates.Thereforeevenmorecareisneededtostorethemsafely.
TocreateaCAcertificateusedtosignclientauthenticationcertificates,run:
arangosynccreateclient-authca\
--cert=my-client-auth-ca.crt--key=my-client-auth-ca.key
Makesuretoprotectandstorebothgeneratedfiles(my-client-auth-ca.crt&my-client-auth-ca.key)inasafeplace.Note:CAcertificateshaveamuchlongerlifetimethannormalcertificates.Thereforeevenmorecareisneededtostorethemsafely.
Renewingcertificates
Allcertificateshavemetainformationinthemthelimittheiruseinfunction,target&lifetime.Acertificatecreatedforclientauthentication(function)cannotbeusedasaTLSservercertificate(sameistrueforthereverse).Acertificateforhostmyserver(target)cannotbeusedforhostanotherserver.AcertficiatethatisvaliduntilOctober2017(limetime)cannotbeusedafterOctober2017.
Ifanythingchangesinfunction,targetorlifetimeyouneedanewcertificate.
Theprocedureforcreatingarenewedcertificateisthesameasforcreatinga"first"certificate.Aftercreatingtherenewedcertificatetheprocess(es)usingthemhavetobeupdated.Thismeanrestartingthem.AllArangoSynccomponentsaredesignedtosupportstoppingandstartingsingleinstances,butdonotrestartmorethan1instanceatthesametime.Assoonas1instancehasbeenrestarted,giveitsometimeto"catchup"beforerestartingthenextinstance.
MultipleDatacenters
316
AdvancedTopics
Incontrasttotheothertopicsinthischapterthatstrivetogetyousimplysetupinpreparedenvironments,Thefollowingchaptersdescribewhatsgoingonunderthehoodindetails,thecomponentsofArangoDBClusters,andhowthey'reputtogether:
RunningalocaltestsetupRunningadistributedsetupRunninginDocker
AdvancedTopics
317
LaunchingArangoDB'sstandalone"agency"
MultipleArangoDBinstancescanbedeployedasafault-tolerantdistributedstatemachine.
Whatisafault-tolerantstatemachineinthefirstplace?
Inmanyservicedeploymentsconsistingofarbitrarycomponentsdistributedovermultiplemachinesoneisfacedwiththechallengeofcreatingadependablecentralisedknowledgebaseorconfiguration.Implementationofsuchaserviceturnsouttobeoneofthemostfundamentalproblemsininformationengineering.Whileitmayseemasiftherealisationofsuchaserviceiseasilyconceivable,dependablityformulatesaparadoxononcomputernetworksperse.Ontheonehand,oneneedsadistributedsystemtoavoidasinglepointoffailure.Ontheotherhand,onehastoestablishconsensusamongthecomputersinvolved.
Consensusisthekeywordhereanditsrealisationonanetworkprovestobefarfromtrivial.Manypapersandconferenceproceedingshavediscussedandevaluatedthiskeychallenge.Twoalgorithms,historicallyfarapart,havebecomewidelypopular,namelyPaxosanditsderivativesandRaft.Discussingthemandtheirdifferences,althoughhighlyenjoyable,mustremainfarbeyondthescopeofthisdocument.Findthereferencestothemainpublicationsatthebottomofthispage.
AtArangoDB,wedecidedtoimplementRaftasitisarguablytheeasiertounderstandandthusimplement.Insimpleterms,Raftguaranteesthatalinearstreamoftransactions,isreplicatedinrealtimeamongagroupofmachinesthroughanelectedleader,whointurnmusthaveaccesstoandprojectleadershipuponanoverallmajorityofparticipatinginstances.InArangoDBweliketocalltheentiretyofthecomponentsofthereplicatedtransactionlog,thatisthemachinesandtheArangoDBinstances,whichconstitutethereplicatedlog,theagency.
Startup
Theagencymustconsistsofanoddnumberofagentsinordertobeabletoestablishanoverallmajorityandsomemeansfortheagentstobeabletofindoneanotheratstartup.
Themostobviouswaywouldbetoinformallagentsoftheaddressesandportsoftherest.Thishowever,ismoreinformationthanneeded.Forexample,itwouldsuffice,ifallagentswouldknowtheaddressandportofthenextagentinacyclicfashion.Anotherstraitforwardsolutionwouldbetoinformallagentsoftheaddressandportofsaythefirstagent.
Clearlyallcases,whichwouldformdisjunctsubsetsofagentswouldbreakorintheleastimpairthefunctionalityoftheagency.Fromthereontheagentswillgossipthemissinginformationabouttheirpeers.
Typically,oneachievesfairlyhighfault-tolerancewithlow,oddnumberofagentswhilekeepingthenecessarynetworktrafficataminimum.Itseemsthatthetypicalagencysizewillbeinrangeof3to7agents.
Thebelowcommandsstartupa3-hostagencyononephysical/logicalboxwithports8529,8530and8531fordemonstrationpurposes.Theadressofthefirstinstance,port8529,isknowntotheothertwo.Afteratmost2roundsofgossipping,thelast2agentswillhaveacompletepictureoftheirsurroundingandpersistitforthenextrestart.
./build/bin/arangod--agency.activatetrue--agency.size3--agency.my-addresstcp://localhost:8529--server.authenticationfal
se--server.endpointtcp://0.0.0.0:8529agency-8529
./build/bin/arangod--agency.activatetrue--agency.size3--agency.endpointtcp://localhost:8529--agency.my-addresstcp://loc
alhost:8530--server.authenticationfalse--server.endpointtcp://0.0.0.0:8530agency-8530
./build/bin/arangod--agency.activatetrue--agency.size3--agency.endpointtcp://localhost:8529--agency.my-addresstcp://loc
alhost:8531--server.authenticationfalse--server.endpointtcp://0.0.0.0:8531agency-8531
Theparameteragency.endpointisthekeyingredientforthesecondandthirdinstancestofindthefirstinstanceandthusformacompleteagency.Pleaserefertothetheshell-scriptscripts/startStandaloneAgency.shongithuborinthesourcedirectory.
Key-value-storeAPI
Theagencyshouldbeupandrunningwithinacoupleofseconds,duringwhichtheinstanceshavegossipedtheirwayintoknowingtheotheragentsandelectedaleader.ThepublicAPIcanbecheckedforthestateoftheconfiguration:
curl-slocalhost:8529/_api/agency/config
StandaloneAgency
318
{
"term":1,
"leaderId":"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98",
"lastCommitted":1,
"lastAcked":{
"ac129027-b440-4c4f-84e9-75c042942171":0.21,
"c54dbb8a-723d-4c82-98de-8c841a14a112":0.21,
"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98":0
},
"configuration":{
"pool":{
"ac129027-b440-4c4f-84e9-75c042942171":"tcp://localhost:8531",
"c54dbb8a-723d-4c82-98de-8c841a14a112":"tcp://localhost:8530",
"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98":"tcp://localhost:8529"
},
"active":[
"ac129027-b440-4c4f-84e9-75c042942171",
"c54dbb8a-723d-4c82-98de-8c841a14a112",
"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98"
],
"id":"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98",
"agencysize":3,
"poolsize":3,
"endpoint":"tcp://localhost:8529",
"minping":0.5,
"maxping":2.5,
"supervision":false,
"supervisionfrequency":5,
"compactionstepsize":1000,
"supervisiongraceperiod":120
}
}
Tohighlightsomedetailsintheaboveoutputlookfor"term"and"leaderId".BotharekeyinformationaboutthecurrentstateoftheRaftalgorithm.Youmayhavenotedthatthefirstelectiontermhasestablishedarandomleaderfortheagency,whoisinchargeofreplicationofthestatemachineandforallexternalreadandwriterequestsuntilsuchtimethattheprocessgetsisolatedfromtheothertwosubsequenctlylosingitsleadership.
ReadandWriteAPIs
Generally,allreadandwriteaccessesaretransactionsmoreoveranyreadandwriteaccessmayconsistofmultiplesuchtransactionsformulatedasarraysofarraysinJSONdocuments.
Readtransaction
Anagencystartedfromscratchwilldealwiththesimplestqueryasfollows:
curl-Llocalhost:8529/_api/agency/read-d'[["/"]]'
[{}]
Theaboverequestforanemptykeyvaluestorewillreturnwithanemptydocument.Theinnerarraybracketswillaggregatearesultfrommultiplesourcesinthekey-value-storewhiletheouterarraywilldelivermultiplesuchaggregatedresults.Alsonotethe-Lcurlflag,whichallowstherequesttofollowredirectstothecurrentleader.
Considerthefollowingkey-value-store:
{
"baz":12,
"corge":{
"e":2.718281828459045,
"pi":3.14159265359
},
"foo":{
"bar":"HelloWorld"
},
StandaloneAgency
319
"qux":{
"quux":"HelloWorld"
}
}
Thefollowingarrayofreadtransactionswillyield:
curl-Llocalhost:8529/_api/agency/read-d'[["/foo","/foo/bar","/baz"],["/qux"]]'
[
{
"baz":12,
"foo":{
"bar":"HelloWorld"
}
},
{
"qux":{
"quux":"HelloWorld"
}
}
]
Notethattheresultisanarrayoftworesultsforthefirstandsecondreadtransactionsfromaboveaccordingly.Alsonotethattheresultsfromthefirstreadtransactionareaggregatedinto
{
"baz":12,
"foo":{
"bar":"HelloWorld"
}
}
Theaggregationisperformedon2levels:
1. /foo/bariseliminatedasasubsetof/foo2. Theresultsfrom/fooand/bararejoined
Thewordtransactionmeansherethatitisguaranteedthatallaggregationshappeninquasi-realtimeandthatnowriteaccesscouldhavehappenedinbetween.
Btw,thesametransactiononthevirginkey-valuestorewouldproduce[{},{}]
WriteAPI:
ThewriteAPImustunfortunatelybealittlemorecomplex.MultipleroadsleadtoRome:
curl-Llocalhost:8529/_api/agency/write-d'[[{"/foo":{"op":"push","new":"bar"}}]]'
curl-Llocalhost:8529/_api/agency/write-d'[[{"/foo":{"op":"push","new":"baz"}}]]'
curl-Llocalhost:8529/_api/agency/write-d'[[{"/foo":{"op":"push","new":"qux"}}]]'
and
curl-Llocalhost:8529/_api/agency/write-d'[[{"foo":["bar","baz","qux"]}]]'
areequivalentforexampleandwillcreateandfillanarrayat/foo.Here,again,theoutermostarrayisthecontainerforthetransactionarrays.
WedocumententacompleteguideoftheAPIintheAPIsection.
StandaloneAgency
320
LaunchinganArangoDBclusterfortesting
AnArangoDBclusterconsistsofseveralrunningtasks(orserverprocesses)whichformthecluster.ArangoDBitselfwon'tstartormonitoranyofthesetasks.Soitwillneedsomekindofsupervisorwhichismonitoringandstartingthesetasks.ForproductionusagewerecommendusingApacheMesosastheclustersupervisor.
HoweverstartingaclustermanuallyispossibleandisaveryeasymethodtogetafirstimpressionofwhatanArangoDBclusterlookslike.
Theeasiestwaytostartalocalclusterfortestingpurposesistorunscripts/startLocalCluster.shfromacloneofthesourcerepositoryaftercompilingArangoDBfromsource(seeinstructionsinthefileREADME_maintainers.mdintherepository.Thiswillstart1Agency,2DBServersand1Coordinator.Tostoptheclusterissuescripts/stopLocalCluster.sh.
ThissectionwilldiscusstherequiredparametersforeveryroleinanArangoDBcluster.BesuretoreadtheArchitecturedocumentationtogetabasicunderstandingoftheunderlyingarchitectureandtheinvolvedrolesinanArangoDBcluster.
Inthefollowingsectionswewillgothroughtherelevantoptionsperrole.
Agency
Tostartupanagencyyoufirsthavetoactivateit.Thisisdonebyproviding--agency.activatetrue.
Tostartuptheagencyinitsfaulttolerantmodesetthe--agency.sizeto3.Youwillthenhavetoprovideatleast3agentsbeforetheagencywillstartoperation.
Duringinitializationtheagentshavetofindeachother.Todosoprovideatleastonecommon--agency.endpoint.Theagentswillthencoordinatestartupthemselves.Theywillannouncethemselveswiththeirexternaladdresswhichmaybespecifiedusing--agency.my-address.ThisisrequiredinbridgeddockersetupsorNATedenvironments.
Soinsummarythisiswhatyourstartupmightlooklike:
Cluster:Localtestsetups
321
arangod--server.endpointtcp://0.0.0.0:5001--agency.my-address=tcp://127.0.0.1:5001--server.authenticationfalse--agency.ac
tivatetrue--agency.size3--agency.endpointtcp://127.0.0.1:5001--agency.supervisiontrue--database.directoryagency1&
arangod--server.endpointtcp://0.0.0.0:5002--agency.my-address=tcp://127.0.0.1:5002--server.authenticationfalse--agency.ac
tivatetrue--agency.size3--agency.endpointtcp://127.0.0.1:5001--agency.supervisiontrue--database.directoryagency2&
arangod--server.endpointtcp://0.0.0.0:5003--agency.my-address=tcp://127.0.0.1:5003--server.authenticationfalse--agency.ac
tivatetrue--agency.size3--agency.endpointtcp://127.0.0.1:5001--agency.supervisiontrue--database.directoryagency3&
Ifyouarehappywithasingleagent,thensimplyuseasinglecommandlikethis:
arangod--server.endpointtcp://0.0.0.0:5001--server.authenticationfalse--agency.activatetrue--agency.size1--agency.endp
ointtcp://127.0.0.1:5001--agency.supervisiontrue--database-directoryagency1&
Furthermore,inthefollowingsectionswhen--cluster.agency-addressisusedmultipletimestospecifyallthreeagentaddresses,justuseasingleoption--cluster.agency.addresstcp://127.0.0.1:5001instead.
CoordinatorsandDBServers
Thesetworolesshareacommonsetofrelevantoptions.Firstyoushouldspecifytheroleusing--cluster.my-role.ThiscaneitherbePRIMARY(adatabaseserver)orCOORDINATOR.Furthermoreprovidetheexternalendpoint(IPandport)ofthetaskvia--cluster.my-address.
Thefollowingisafull-exampleofwhatitmightlooklike:
arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8529--cluster.my-addresstcp://127.0.0.1:8529--cluster.
my-rolePRIMARY--cluster.agency-endpointtcp://127.0.0.1:5001--cluster.agency-endpointtcp://127.0.0.1:5002--cluster.agency-
endpointtcp://127.0.0.1:5003--database.directoryprimary1&
arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8530--cluster.my-addresstcp://127.0.0.1:8530--cluster.
my-rolePRIMARY--cluster.agency-endpointtcp://127.0.0.1:5001--cluster.agency-endpointtcp://127.0.0.1:5002--cluster.agency-
endpointtcp://127.0.0.1:5003--database.directoryprimary2&
arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8531--cluster.my-addresstcp://127.0.0.1:8531--cluster.
my-roleCOORDINATOR--cluster.agency-endpointtcp://127.0.0.1:5001--cluster.agency-endpointtcp://127.0.0.1:5002--cluster.age
ncy-endpointtcp://127.0.0.1:5003--database.directorycoordinator&
Noteinparticularthattheendpointdescriptionsgivenunder--cluster.my-addressand--cluster.agency-endpointmustnotusetheIPaddress0.0.0.0becausetheymustcontainanactualaddressthatcanberoutedtothecorrespondingserver.The0.0.0.0in--server.endpointsimplymeansthattheserverbindsitselftoallavailablenetworkdeviceswithallavailableIPaddresses.
UponregisteringwiththeagencyduringstartuptheclusterwillassignanIDtoeveryserver.ThegeneratedIDwillbeprintedouttothelogorcanbeaccessedviathehttpAPIbycallinghttp://server-address/_admin/server/id.
YouhavenowlaunchedacompleteArangoDBclusterandcancontactitscoordinatorattheendpointtcp://127.0.0.1:8531,whichmeansthatyoucanreachthewebUIunderhttp://127.0.0.1:8531.
Cluster:Localtestsetups
322
LaunchinganArangoDBclusteronmultiplemachines
Essentially,onecanusethemethodfromtheprevioussectiontostartanArangoDBclusteronmultiplemachinesaswell.Theonlychangesarethatonehastoreplacealllocaladdresses127.0.0.1bytheactualIPaddressofthecorrespondingserver.
IfweassumethatyouwanttostartyouArangoDBclusteronthreedifferentmachineswithIPaddresses
192.168.1.1
192.168.1.2
192.168.1.3
thenthecommandsyouhavetouseare(youcanusehostnamesiftheycanberesolvedtoIPaddressesonallmachines):
On192.168.1.1:
sudoarangod--server.endpointtcp://0.0.0.0:5001--agency.my-addresstcp://192.168.1.1:5001--server.authenticationfalse--ag
ency.activatetrue--agency.size3--agency.supervisiontrue--database.directoryagency
On192.168.1.2:
sudoarangod--server.endpointtcp://0.0.0.0:5001--agency.my-addresstcp://192.168.1.2:5001--server.authenticationfalse--ag
ency.activatetrue--agency.size3--agency.supervisiontrue--database.directoryagency
On192.168.1.3:
sudoarangod--server.endpointtcp://0.0.0.0:5001--agency.my-addresstcp://192.168.1.3:5001--server.authenticationfalse--ag
ency.activatetrue--agency.size3--agency.endpointtcp://192.168.1.1:5001--agency.endpointtcp://192.168.1.2:5001--agency.e
ndpointtcp://192.168.1.3:5001--agency.supervisiontrue--database.directoryagency
On192.168.1.1:
sudoarangod--server.authentication=false--server.endpointtcp://0.0.0.0:8529--cluster.my-addresstcp://192.168.1.1:8529--c
luster.my-rolePRIMARY--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agency-endpointtcp://192.168.1.2:5001--clus
ter.agency-endpointtcp://192.168.1.3:5001--database.directoryprimary1&
On192.168.1.2:
sudoarangod--server.authentication=false--server.endpointtcp://0.0.0.0:8530--cluster.my-addresstcp://192.168.1.2:8530--c
luster.my-rolePRIMARY--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agency-endpointtcp://192.168.1.2:5001--clus
ter.agency-endpointtcp://192.168.1.3:5001--database.directoryprimary2&
On192.168.1.3:
arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8531--cluster.my-addresstcp://192.168.1.3:8531--cluste
r.my-roleCOORDINATOR--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agency-endpointtcp://192.168.1.2:5001--clust
er.agency-endpointtcp://192.168.1.3:5001--database.directorycoordinator&
Obviously,itwouldnolongerbenecessarytousedifferentportnumbersondifferentservers.Wehavechosentokeepallportnumbersincomparisontothelocalsetuptominimizethenecessarychanges.
Afterhavingswallowedtheselongishcommands,wehopethatyouappreciatethesimplicityofthesetupwithApacheMesosandDC/OS.
Cluster:Processes
323
ArangoDBClusterandDocker
Networking
AbitofextracarehastobeinvestedduetothewayinwhichDockerisolatesitsnetwork.Bydefaultitfullyisolatesthenetworkandbydoingsoanendpointlike--server.endpointtcp://0.0.0.0:8529willonlybindtoallinterfacesinsidetheDockercontainerwhichdoesnotincludeanyexternalinterfaceonthehostmachine.ThismaybesufficientifyoujustwanttoaccessitlocallybutincaseyouwanttoexposeittotheoutsideyoumustfacilitateDockersportforwardingusingthe-pcommandlineoption.BesuretochecktheofficialDockerdocumentation.
Tosimplymakearangodbavailableonallhostinterfacesonport8529:
dockerrun-p8529:8529-eARANGO_NO_AUTH=1arangodb
AnotherpossibilityistostartDockervianetworkmodehost.Thisispossiblebutgenerallynotrecommended.TodoitanywaychecktheDockerdocumentationfordetails.
DockerandClustertasks
TostarttheclusterviaDockerisbasicallythesameasstartinglocallyoronmultiplemachines.Howeverjustlikewiththesinglenetworkingimagewewillfacenetworkingissues.Youcansimplyusethe-pflagtomaketheindividualtaskavailableonthehostmachineoryoucoulduseDocker'slinkstoenabletaskintercommunication.
PleasenotethattherearesomeflagsthatspecifyhowArangoDBcanreachataskfromtheoutside.Theseareveryimportantandbuiltforthisexactusecase.Anexampleconfigurationmightlooklikethis:
dockerrun-eARANGO_NO_AUTH=1-p192.168.1.1:10000:8529arangodb/arangodbarangod--server.endpointtcp://0.0.0.0:8529--clust
er.my-addresstcp://192.168.1.1:10000--cluster.my-rolePRIMARY--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agen
cy-endpointtcp://192.168.1.2:5002--cluster.agency-endpointtcp://192.168.1.3:5003
ThiswillstartaprimaryDBserverwithinaDockercontainerwithanisolatednetwork.WithintheDockercontaineritwillbindtoallinterfaces(thiswillbe127.0.0.1:8529andsomeinternalDockeriponport8529).Bysupplying-p192.168.1.1:10000:8529weareestablishingaportforwardingfromourlocalIP(192.168.1.1port10000inthisexample)toport8529insidethecontainer.Withinthecommandwearetellingarangodhowitcanbereachedfromtheoutside--cluster.my-addresstcp://192.168.1.1:10000.ThisinformationwillbeforwardedtotheagencysothattheothertasksinyourclustercanseehowthisparticularDBServermaybereached.
Cluster:Docker
324
AdministrationMostadministrationcanbemanagedusingthearangosh.
FilesystemsAsonewouldexpectforadatabase,werecommendalocallymountedfilesystems.
NFSorsimilarnetworkfilesystemswillnotwork.
OnLinuxwerecommendtheuseofext4fs,onWindowsNTFSandonMacOSHFS+.
WerecommendtonotuseBTRFSonLinux.ItisknowntonotworkwellinconjunctionwithArangoDB.WeexperiencedthatArangoDBfaceslatencyissuesonaccessingitsdatabasefilesonBTRFSpartitions.InconjunctionwithBTRFSandAUFSwealsosawdatalossonrestart.
Administration
325
WebInterfaceArangoDBcomeswithabuilt-inwebinterfaceforadministration.Theinterfacediffersforstandaloneinstancesandclustersetups.
Standalone:
Cluster:
WebInterface
326
WebInterface
327
DashboardTheDashboardtabprovidesstatisticswhicharepolledregularlyfromtheArangoDBserver.
RequestsStatistics:
RequestspersecondRequesttypesNumberofclientconnectionsTransfersizeTransfersize(distribution)AveragerequesttimeAveragerequesttime(distribution)
SystemResources:
NumberofthreadsMemoryVirtualsizeMajorpagefaultsUsedCPUtime
Replication:
ReplicationstateTotalsTicksProgress
Dashboard
328
ClusterTheclustersectiondisplaysstatisticsaboutthegeneralclusterperformance.
Statistics:
AvailableandmissingcoordinatorsAvailableandmissingdatabaseserversMemoryusage(percent)CurrentconnectionsData(bytes)HTTP(bytes)Averagerequesttime(seconds)
Nodes
Overview
Theoverviewshowsavailableandmissingcoordinatorsanddatabaseservers.
Cluster
329
Functions:
CoordinatorDashboard:ClickonaCoordinatorwillopenastatisticsdashboard.
Information(Coordinator/Databaseservers):
NameEndpointLastHeartbeatStatusHealth
Shards
Theshardsectiondisplaysallavailableshardedcollections.
Cluster
330
Functions:
MoveShardLeader:Clickonaleaderdatabaseofashardserverwillopenamovesharddialog.Shardscanbetransferredtoallavailabledatabasservers,excepttheleadingdatabaseserveroranavailablefollower.MoveShardFollower:Clickonafollowerdatabaseofashardwillopenamovesharddialog.Shardscanbetransferredtoallavailabledatabasservers,excepttheleadingdatabaseserveroranavailablefollower.RebalanceShards:Anewdatabaseserverwillnothaveanyshards.Withtherebalancefunctionalitytheclusterwillstarttorebalanceshardsincludingemptydatabaseservers.
Information(collection):
ShardLeader(greenstate:synciscomplete)Followers
Cluster
331
CollectionsThecollectionssectiondisplaysallavailablecollections.Fromhereyoucancreatenewcollectionsandjumpintoacollectionfordetails(clickonacollectiontile).
Functions:
A:TogglefilterpropertiesB:SearchcollectionbynameD:CreatecollectionC:FilterpropertiesH:Showcollectiondetails(clicktile)
Information:
E:CollectiontypeF:Collectionstate(unloaded,loaded,...)G:Collectionname
Collection
Collections
332
Therearefourviewcategories:
1. Content:
CreateadocumentDeleteadocumentFilterdocumentsDownloaddocumentsUploaddocuments
2. Indices:
CreateindicesDeleteindices
3. Info:
Detailedcollectioninformationandstatistics4. Settings:
Configurename,journalsize,indexbuckets,waitforsyncDeletecollectionTruncatecollectionUnload/LoadcollectionSavemodifedproperties(name,journalsize,indexbuckets,waitforsync)
Additionalinformation:
Uploadformat:
I.Line-wise
{"_key":"key1",...}
{"_key":"key2",...}
II.JSONdocumentsinalist
[
{"_key":"key1",...},
{"_key":"key2",...}
Collections
333
]
Collections
334
DocumentThedocumentsectionoffersaeditorwhichletyoueditdocumentsandedgesofacollection.
Functions:
EditdocumentSavedocumentDeletedocmentSwitchbetweenTree/Code-ModeCreateanewdocument
Information:
Displays:_id,_rev,_keyproperties
Document
335
QueryViewThequeryviewoffersyouthreedifferentsubviews:
EditorRunningQueriesSlowQueryHistory
AQLQueryEditor
ThewebinterfaceoffersaAQLQueryEditor:
Theeditorissplitintotwoparts,thequeryeditorpaneandthebindparameterpane.
Theleftpaneisyourregularqueryinputfield,whereyoucaneditandthenexecuteorexplainyourqueries.Bydefault,theenteredbindparameterwillautomaticallyberecognizedandshowninthebindparametertableintherightpane,whereyoucaneasilyeditthem.
Theinputfieldsareequippedwithtypedetection.Thismeansyoudon'thavetousequotemarksaroundstring,justwritethemas-is.Numberswillbetreatedasnumbers,trueandfalseasbooleans,nullasnull-typevalue.Squarebracketscanbeusedtodefinearrays,andcurlybracesforobjects(keysandvalueshavetobesurroundedbydoublequotes).Thiswillmostlybewhatyouwant.Butifyouwanttoforcesomethingtobetreatedasstring,usequotationmarksforthevalue:
123//interpretedasnumber
"123"//interpretedasstring
["foo","bar",123,true]//interpretedasarray
['foo','bar',123,true]//interpretedasstring
Queries
336
IfyouareusedtoworkwithJSON,youmaywanttoswitchthebindparametereditortoJSONmodebyclickingontheupperrighttogglebutton.YoucantheneditthebindparametersinrawJSONformat.
CustomQueries
TosavethecurrentqueryusetheSavebuttoninthetopleftcorneroftheeditororusetheshortcut(seebelow).
BypressingtheQueriesbuttoninthetopleftcorneroftheeditoryouactivatethecustomqueriesview.Hereyoucanselectapreviouslystoredcustomqueryoroneofourqueryexamples.
Clickonaquerytitletogetacodepreview.Inaddition,thereareactionbuttonsto:
CopytoeditorExplainqueryRunqueryDeletequery
Forthebuilt-inexamplequeries,thereisonlyCopytoeditoravailable.
ToexportorimportqueriestoandfromJSONyoucanusethebuttonsontheright-handside.
Result
Queries
337
Eachqueryyouexecuteorexplainopensupanewresultbox,soyouareabletofireupmultiplequeriesandviewtheirresultsatthesametime.Everyqueryresultboxgivesyoudetailedqueryinformationandofcoursethequeryresultitself.Theresultboxescanbedismissedindividually,oraltogetherusingtheRemoveresultsbutton.ThetogglebuttoninthetoprightcornerofeachboxswitchesbackandforthbetweentheResultandAQLquerywithbindparameters.
Spotlight
Thespotlightfeatureopensupamodalview.ThereyoucanfindallAQLkeywords,AQLfunctionsandcollections(filteredbytheirtype)tohelpyoutobemoreproductiveinwritingyourqueries.Spotlightcanbeopenedbythemagicwandiconinthetoolbarorviashortcut(seebelow).
AQLEditorShortcuts
Queries
338
Ctrl/Cmd+ReturntoexecuteaqueryCtrl/Cmd+Shift+ReturntoexplainaqueryCtrl/Cmd+Shift+StosavethecurrentqueryCtrl/Cmd+Shift+CtotogglecommentsCtrl+SpacetoopenupthespotlightsearchCtrl+Cmd+ZtoundolastchangeCtrl+Cmd+Shift+Ztoredolastchange
RunningQueries
TheRunningQueriestabgivesyouacompactoverviewofallrunningqueries.Byclickingtheredminusbutton,youcanaborttheexecutionofarunningquery.
SlowQueryHistory
Queries
339
TheSlowQueryHistorytabgivesyouacompactoverviewofallpastslowqueries.
Queries
340
GraphsTheGraphstabprovidesaviewerfacilityforgraphdatastoredinArangoDB.ItallowsbrowsingArangoDBgraphsstoredinthe_graphssystemcollectionoragraphconsistingofanarbitraryvertexandedgecollection.
Pleasenotethatthegraphviewerrequirescanvas(optional:webgl)supportinyourbrowser.EspeciallyInternetExplorerbrowsersolderthanversion9arelikelytonotsupportthis.
GraphViewer
Graphs
341
TopToolbarFunctions:
Loadfullgraph(Alsonodeswithoutconnectionswillbedrawn.Usefulduringgraphmodelingsetup)TakeagraphscreenshotStartfullscreenmodeOpengraphoptionsmenu
DefaultContextMenu(mouse-clickbackground):
AddanewnodeClosevisiblecontextmenu(s)
NodeContextMenu(mouse-clicknode):
DeletenodeEditnodeExpandnode(Showallboundedges)Drawedge(Connectwithanothernode)Setasstartnode(TheGraphwillrerenderstartingtheselectednodeandgivenoptions(graphoptionsmenu))
EdgeContextMenu(mouse-clickedge):
EditedgeDeleteedge
EdgeHighlighting(right-mouse-clicknode):
Highlightalledgesconnectedtothenode(right-clickatthebackgroundwillremovehighlighting)
Graphs
342
GraphViewerOptionsGraphOptionsMenu:
Startnode(string-validnodeidorspaceseperatedlistofid's):Heartofyourgraph.Renderingandtraversingwillstartfromhere.Emptyvaluemeans:arandomstartingpointwillbeused.Layout:Differentgraphlayoutingalgoritms.Nooverlap(optimal:biggraph),forcelayout(optimal:mediumgraph),fruchtermann(optimal:littletomediumgraph).Renderer:Canvasmodeallowsediting.WebGLcurrentlyoffersonlydisplaymode(alotfasterwithmuchnodes/edges).Searchdepth(number):Searchdepthwhichisstartingfromyourstartnode.Limit(number):Limitnodescount.Ifemptyorzero,nolimitisset.
NodesOptionsMenu:
Label(string):Nodeswillbelabeledbythisattribute.Ifnodeattributeisnotfound,nolabelwillbedisplayed.AddCollectionName:Thisappendsthecollectionnametothelabel,ifitexists.ColorByCollections:Shouldnodesbecolorizedbytheircollection?Ifenabled,nodecolorandnodecolorattributewillbeignored.Color:Defaultnodecolor.ColorAttribute(string):Ifanattributeisgiven,nodeswillthenbecolorizedbytheattribute.Thissettingignoresdefaultnodecolorifset.SizeByConnections:Shouldnodesbesizedbytheiredgescount?Ifenabled,nodesizingattributewillbeignored.SizingAttribute(number):Defaultnodesize.Numericvalue>0.
EdgesOptionsMenu:
Label(string):Edgeswillbelabeledbythisattribute.Ifedgeattributeisnotfound,nolabelwillbedisplayed.AddCollectionName:Thisappendsthecollectionnametothelabel,ifitexists.ColorByCollections:Shouldedgesbecolorizedbytheircollection?Ifenabled,edgecolorandedgecolorattributewillbeignored.Color:Defaultedgecolor.ColorAttribute(string):Ifanattributeisgiven,edgeswillthenbecolorizedbytheattribute.Thissettingignoresdefaultnodecolorifset.Type:Therendereroffersmultipletypesofrendering.Theyonlydifferintheirdisplaystyle,exceptforthetype'curved'.Thecurvedtypeallowstodisplaymorethanoneedgesbetweentwonodes.
Graphs
343
Graphs
344
ServicesTheservicessectiondisplaysallinstalledfoxxapplications.Youcancreatenewservicesorgointoadetailedviewofachoosenservice.
CreateService
Therearefourdifferentpossibilitiestocreateanewservice:
1. Createserviceviazipfile2. Createserviceviagithubrepository3. CreateserviceviaofficialArangoDBstore4. Createablankservicefromscratch
Services
345
ServiceView
Thissectionoffersseveralinformationaboutaspecificservice.
Therearefourviewcategories:
1. Info:
Displaysname,shortdescription,license,version,mode(production,development)Offersabuttontogototheservicesinterface(ifavailable)
Services
346
2. Api:
DisplayAPIasSwaggerUIDisplayAPIasRAWJSON
3. Readme:
Displaystheservicesmanual(ifavailable)4. Settings:
DownloadserviceaszipfileRunservicetests(ifavailable)Runservicescripts(ifavailable)Configuredependencies(ifavailable)Changeserviceparameters(ifavailable)Changemode(production,development)ReplacetheserviceDeletetheservice
Services
347
ManagingUsersintheWebInterfaceArangoDBusersaregloballystoredinthe_systemdatabaseandcanonlybemanangedwhileloggedontothisdatabase.ThereyoucanfindtheUserssection:
General
SelectausertobringuptheGeneraltabwiththeusername,nameandactivestatus,aswellasoptionstodeletetheuserorchangethepassword.
Users
348
Permissions
SelectauserandgotothePermissionstab.Youwillseealistofdatabasesandtheircorrespondingdatabaseaccesslevelforthatuser.
Users
349
Pleasenotethatserveraccesslevelfollowsfromtheaccesslevelonthedatabase_system.Furthermore,thedefaultdatabaseaccesslevelforthisuserappearintheartificialrowwiththedatabasename*.
Belowthistableisanotheroneforthecollectioncategoryaccesslevels.Atfirst,itshowsthelistofdatabases,too.Ifyouclickonadatabase,thelistofcollectionsinthatdatabasewillbeopenandyoucanseethedefinedcollectionaccesslevelsforeachcollectionofthatdatabase(whichcanbeallunselectedwhichmeansthatnothingisexplicitlyset).Thedefaultaccesslevelsforthisuseranddatabaseappearintheartificialrowwiththecollectionname*.
AlsoseeManagingUsersaboutaccesslevels.
Users
350
LogsThelogssectiondisplaysallavailablelogentries.Logentriesarefilterablebytheirlogleveltypes.
Functions:
Filterlogentriesbyloglevel(all,info,error,warning,debug)
Information:
LoglevelDateMessage
Logs
351
ArangoDBShellIntroductionTheArangoDBshell(arangosh)isacommand-linetoolthatcanbeusedforadministrationofArangoDB,includingrunningad-hocqueries.
ThearangoshbinaryisshippedwithArangoDB.ItoffersaJavaScriptshellenvironmentprovidingaccesstotheArangoDBserver.Arangoshcanbeinvokedlikethis:
unix>arangosh
BydefaultarangoshwilltrytoconnecttoanArangoDBserverrunningonserverlocalhostonport8529.Itwillusetheusernamerootandanemptypasswordbydefault.Additionallyitwillconnecttothedefaultdatabase(_system).Allthesedefaultscanbechangedusingthefollowingcommand-lineoptions:
--server.database:nameofthedatabasetoconnectto--server.endpoint:endpointtoconnectto--server.username:databaseusername--server.password:passwordtousewhenconnecting--server.authentication:whetherornottouseauthentication
Forexample,toconnecttoanArangoDBserveronIP192.168.173.13onport8530withtheuserfooandusingthedatabasetest,use:
unix>arangosh\
--server.endpointtcp://192.168.173.13:8530\
--server.usernamefoo\
--server.databasetest\
--server.authenticationtrue
arangoshwillthendisplayapasswordpromptandtrytoconnecttotheserverafterthepasswordwasentered.
Tochangethecurrentdatabaseaftertheconnectionhasbeenmade,youcanusethedb._useDatabase()commandinarangosh:
arangosh>db._createDatabase("myapp");
true
arangosh>db._useDatabase("myapp");
true
arangosh>db._useDatabase("_system");
true
arangosh>db._dropDatabase("myapp");
true
Togetalistofavailablecommands,arangoshprovidesahelp()function.Callingitwilldisplayhelpfulinformation.
arangoshalsoprovidesauto-completion.Additionalinformationonavailablecommandsandmethodsisthusprovidedbytypingthefirstfewlettersofavariableandthenpressingthetabkey.Itisrecommendtotrythiswithenteringdb.(withoutpressingreturn)andthenpressingtab.
Bytheway,arangoshprovidesthedbobjectbydefault,andthisobjectcanbeusedforswitchingtoadifferentdatabaseandmanagingcollectionsinsidethecurrentdatabase.
Foralistofavailablemethodsforthedbobject,type
arangosh>db._help();
showexecutionresultsyoucanpastemultiplelinesintoarangosh,giventhefirstlineendswithanopeningbrace:
arangosh>for(vari=0;i<10;i++){
ArangoDBShell
352
........>require("@arangodb").print("Helloworld"+i+"!\n");
........>}
showexecutionresultsToloadyourownJavaScriptcodeintothecurrentJavaScriptinterpretercontext,usetheloadcommand:
require("internal").load("/tmp/test.js")//<-Linux/MacOS
require("internal").load("c:\\tmp\\test.js")//<-Windows
Exitingarangoshcanbedoneusingthekeycombination<CTRL>+Dorbytypingquit<CR>
Escaping
InAQL,escapingisdonetraditionallywiththebackslashcharacter:\.Asseenabove,thisleadstodoublebackslasheswhenspecifyingWindowspaths.Arangoshrequiresanotherlevelofescaping,alsowiththebackslashcharacter.ItaddsuptofourbackslashesthatneedtobewritteninArangoshforasingleliteralbackslash(c:\tmp\test.js):
db._query('RETURN"c:\\\\tmp\\\\test.js"')
Youcanusebindvariablestomitigatethis:
varsomepath="c:\\tmp\\test.js"
db._query(aql`RETURN${somepath}`)
ArangoDBShell
353
ArangoDBShellOutputTheArangoDBshellwillprinttheoutputofthelastevaluatedexpressionbydefault:
arangosh>42*23
966
Inordertopreventprintingtheresultofthelastevaluatedexpression,theexpressionresultcanbecapturedinavariable,e.g.
arangosh>varcalculationResult=42*23
ThereisalsotheprintfunctiontoexplicitlyprintoutvaluesintheArangoDBshell:
arangosh>print({a:"123",b:[1,2,3],c:"test"});
showexecutionresultsBydefault,theArangoDBshellusesaprettyprinterwhenJSONdocumentsareprinted.Thisensuresdocumentsareprintedinahuman-readableway:
arangosh>db._create("five")
arangosh>for(i=0;i<5;i++)db.five.save({value:i})
arangosh>db.five.toArray()
showexecutionresultsWhilethepretty-printerproducesnicelookingresults,itwillneedalotofscreenspaceforeachdocument.Sometimesamoredenseoutputmightbebetter.Inthiscase,theprettyprintercanbeturnedoffusingthecommandstop_pretty_print().
Toturnonprettyprintingagain,usethestart_pretty_print()command.
ShellOutput
354
ArangoDBShellConfigurationarangoshwilllookforauser-definedstartupscriptnamed.arangosh.rcintheuser'shomedirectoryonstartup.Thehomedirectorywilllikelybe/home/<username>/onUnix/Linux,andisdeterminedonWindowsbypeekingintotheenvironmentvariables%HOMEDRIVE%and%HOMEPATH%.
Ifthefile.arangosh.rcispresentinthehomedirectory,arangoshwillexecutethecontentsofthisfileinsidetheglobalscope.
Youcanusethistodefineyourownextravariablesandfunctionsthatyouneedoften.Forexample,youcouldputthefollowingintothe.arangosh.rcfileinyourhomedirectory:
//"var"keywordavoidedintentionally...
//otherwise"timed"wouldnotsurvivethescopeofthisscript
global.timed=function(cb){
console.time("callback");
cb();
console.timeEnd("callback");
};
Thiswillmakeafunctionnamedtimedavailableinarangoshintheglobalscope.
Youcannowstartarangoshandinvokethefunctionlikethis:
timed(function(){
for(vari=0;i<1000;++i){
db.test.save({value:i});
}
});
Pleasekeepinmindthat,ifpresent,the.arangosh.rcfileneedstocontainvalidJavaScriptcode.Ifyouwantanyvariablesintheglobalscopetosurviveyouneedtoomitthevarkeywordforthem.Otherwisethevariableswillonlybevisibleinsidethescriptitself,butnotoutside.
Configuration
355
DetailsabouttheArangoDBShellAftertheserverhasbeenstarted,youcanusetheArangoDBshell(arangosh)toadministratetheserver.Withoutanyarguments,theArangoDBshellwilltrytocontacttheserveronport8529onthelocalhost.FormoreinformationseetheArangoDBShelldocumentation.Youmightneedtosetadditionaloptions(endpoint,usernameandpassword)whenconnecting:
unix>./arangosh--server.endpointtcp://127.0.0.1:8529--server.usernameroot
TheshellwillprintitsownversionnumberandifsuccessfullyconnectedtoaservertheversionnumberoftheArangoDBserver.
Command-LineOptionsUse--helptogetalistofcommand-lineoptions:
unix>./arangosh--help
STANDARDoptions:
--audit-log<string>auditlogfiletosavecommandsandresultsto
--configuration<string>readconfigurationfile
--helphelpmessage
--max-upload-size<uint64>maximumsizeofimportchunks(inbytes)(default:500000)
--no-auto-completedisableautocompletion
--no-colorsdeactivatecolorsupport
--pager<string>outputpager(default:"less-X-R-F-L")
--pretty-printprettyprintvalues
--quietnobanner
--temp.path<string>pathfortemporaryfiles(default:"/tmp/arangodb")
--use-pagerusepager
JAVASCRIPToptions:
--javascript.check<string>syntaxcheckcodeJavaScriptcodefromfile
--javascript.execute<string>executeJavaScriptcodefromfile
--javascript.execute-string<string>executeJavaScriptcodefromstring
--javascript.startup-directory<string>startuppathscontainingtheJavaScriptfiles
--javascript.unit-tests<string>donotstartasshell,rununittestsinstead
--jslint<string>donotstartasshell,runjslintinstead
LOGGINGoptions:
--log.level<string>loglevel(default:"info")
CLIENToptions:
--server.connect-timeout<double>connecttimeoutinseconds(default:3)
--server.authentication<bool>whetherornottouseauthentication(default:true)
--server.endpoint<string>endpointtoconnectto,use'none'tostartwithoutaserver(default:"tcp://127.0
.0.1:8529")
--server.password<string>passwordtousewhenconnecting(leaveemptyforprompt)
--server.request-timeout<double>requesttimeoutinseconds(default:300)
--server.username<string>usernametousewhenconnecting(default:"root")
DatabaseWrappersThedbobjectisavailableinarangoshaswellasonarangodi.e.ifyou'reusingFoxx.Whileitsinterfaceispersistantbetweenthearangoshandthearangodimplementations,itsunderpinningisnot.ThearangodimplementationareJavaScriptwrappersaroundArangoDB'snativeC++implementation,whereasthearangoshimplementationwrapsHTTPaccessestoArangoDB'sRESTfullAPI.
Sowhilethiscodemayproducesimilarresultswhenexecutedinarangoshandarangod,thecpuusageandtimerequiredwillbereallydifferent:
for(i=0;i<100000;i++){
db.test.save({name:{first:"Jan"},count:i});
}
Sincethearangoshversionwillbedoingaround100kHTTPrequests,andthearangodversionwilldirectlywritetothedatabase.
Details
356
Using arangoshviaunixshebangmechanisms
Inunixoperatingsystemsyoucanstartscriptsbyspecifyingtheinterpreterinthefirstlineofthescript.Thisiscommonlycalledshebangorhashbang.Youcanalsodothatwitharangosh,i.e.create~/test.js:
#!/usr/bin/arangosh--javascript.execute
require("internal").print("helloworld")
db._query("FORxINtestRETURNx").toArray()
Notethatthefirstlinehastoendwithablankinordertomakeitwork.MarkitexecutabletotheOS:
#>chmoda+x~/test.js
andfinalytryitout:
#>~/test.js
Details
357
ArangoimpThismanualdescribestheArangoDBimporterarangoimp,whichcanbeusedforbulkimports.
ThemostconvenientmethodtoimportalotofdataintoArangoDBistousethearangoimpcommand-linetool.Itallowsyoutoimportdatarecordsfromafileintoanexistingdatabasecollection.
Itispossibletoimportdocumentkeyswiththedocumentsusingthe_keyattribute.Whenimportingintoanedgecollection,itismandatorythatallimporteddocumentshavethe_fromand_toattributes,andthattheycontainvalidreferences.
Let'sassumeforthefollowingexamplesyouwanttoimportuserdataintoanexistingcollectionnamed"users"ontheserver.
ImportingDataintoanArangoDBDatabase
ImportingJSON-encodedData
Let'sfurtherassumetheimportathandisencodedinJSON.We'llbeusingtheseexampleuserrecordstoimport:
{"name":{"first":"John","last":"Connor"},"active":true,"age":25,"likes":["swimming"]}
{"name":{"first":"Jim","last":"O'Brady"},"age":19,"likes":["hiking","singing"]}
{"name":{"first":"Lisa","last":"Jones"},"dob":"1981-04-09","likes":["running"]}
Toimporttheserecords,allyouneedtodoistoputthemintoafile(withonelineforeachrecordtoimport)andrunthefollowingcommand:
>arangoimp--file"data.json"--typejsonl--collection"users"
Thiswilltransferthedatatotheserver,importtherecords,andprintastatussummary.Toshowtheintermediateprogressduringtheimportprocess,theoption--progresscanbeadded.Thisoptionwillshowthepercentageoftheinputfilethathasbeensenttotheserver.Thiswillonlybeusefulforbigimportfiles.
>arangoimp--file"data.json"--typejson--collectionusers--progresstrue
Itisalsopossibletousetheoutputofanothercommandasaninputforarangoimp.Forexample,thefollowingshellcommandcanbeusedtopipedatafromthecatprocesstoarangoimp:
>catdata.json|arangoimp--file---typejson--collectionusers
Notethatyouhavetouse--file-ifyouwanttouseanothercommandasinputforarangoimp.Noprogresscanbereportedforsuchimportsasthesizeoftheinputwillbeunknowntoarangoimp.
Bydefault,theendpointtcp://127.0.0.1:8529willbeused.Ifyouwanttospecifyadifferentendpoint,youcanusethe--server.endpointoption.Youprobablywanttospecifyadatabaseuserandpasswordaswell.Youcandosobyusingtheoptions--server.usernameand--server.password.Ifyoudonotspecifyapassword,youwillbepromptedforone.
>arangoimp--server.endpointtcp://127.0.0.1:8529--server.usernameroot--file"data.json"--typejson--collection"users"
Notethatthecollection(usersinthiscase)mustalreadyexistortheimportwillfail.Ifyouwanttocreateanewcollectionwiththeimportdata,youneedtospecifythe--create-collectionoption.Notethatbydefaultitwillcreateadocumentcollectionandnoedecollection.
>arangoimp--file"data.json"--typejson--collection"users"--create-collectiontrue
Tocreateanedgecollectioninstead,usethe--create-collection-typeoptionandsetittoedge:
>arangoimp--file"data.json"--collection"myedges"--create-collectiontrue--create-collection-typeedge
Arangoimp
358
Whenimportingdataintoanexistingcollectionitisoftenconvenienttofirstremovealldatafromthecollectionandthenstarttheimport.Thiscanbeachievedbypassingthe--overwriteparametertoarangoimp.Ifitissettotrue,anyexistingdatainthecollectionwillberemovedpriortotheimport.Notethatanyexistingindexdefinitionsforthecollectionwillbepreservedevenif--overwriteissettotrue.
>arangoimp--file"data.json"--typejson--collection"users"--overwritetrue
AstheimportfilealreadycontainsthedatainJSONformat,attributenamesanddatatypesarefullypreserved.Ascanbeseenintheexampledata,thereisnoneedforalldatarecordstohavethesameattributenamesortypes.Recordscanbeinhomogeneous.
Pleasenotethatbydefault,arangoimpwillimportdataintothespecifiedcollectioninthedefaultdatabase(_system).Tospecifyadifferentdatabase,usethe--server.databaseoptionwheninvokingarangoimp.
Thetoolalsosupportsparallelimports,withmultiplethreads.Usingmultiplethreadsmayprovideaspeedup,especiallywhenusingtheRocksDBstorageengine.Tospecifythenumberofparallelthreadsusethe--threadsoption:
>arangoimp--threads4--file"data.json"--typejson--collection"users"
Notethatusingmultiplethreadsmayleadtoanon-sequentialimportoftheinputdata.Datathatappearslaterintheinputfilemaybeimportedearlierthandatathatappearsearlierintheinputfile.Thisisnormallynotaproblembutmaycauseissueswhenwhentherearedatadependenciesorduplicatesintheimportdata.Inthiscase,thenumberofthreadsshouldbesetto1.
JSONinputfileformats
Note:arangoimpsupportstwoformatswhenimportingJSONdatafromafile.Thefirstformatthatwealsousedaboveiscommonlyknownasjsonl).However,incontrasttotheJSONLspecificationitrequirestheinputfiletocontainonecompleteJSONdocumentineachline,e.g.
{"_key":"one","value":1}
{"_key":"two","value":2}
{"_key":"foo","value":"bar"}
...
SoonecouldarguethatthisisonlyasubsetofJSONL.
Theaboveformatcanbeimportedsequentiallybyarangoimp.Itwillreaddatafromtheinputfileinchunksandsenditinbatchestotheserver.Eachbatchwillbeaboutasbigasspecifiedinthecommand-lineparameter--batch-size.
AnalternativeistoputonebigJSONdocumentintotheinputfilelikethis:
[
{"_key":"one","value":1},
{"_key":"two","value":2},
{"_key":"foo","value":"bar"},
...
]
Thisformatallowslinebreakswithintheinputfileasrequired.Thedownsideisthatthewholeinputfilewillneedtobereadbyarangoimpbeforeitcansendthefirstbatch.Thismightbeaproblemiftheinputfileisbig.Bydefault,arangoimpwillallowimportingsuchfilesuptoasizeofabout16MB.
Ifyouwanttoallowyourarangoimpinstancetousemorememory,youmaywanttoincreasethemaximumfilesizebyspecifyingthecommand-lineoption--batch-size.Forexample,tosetthebatchsizeto32MB,usethefollowingcommand:
>arangoimp--file"data.json"--typejson--collection"users"--batch-size33554432
Pleasealsonotethatyoumayneedtoincreasethevalueof--batch-sizeifasingledocumentinsidetheinputfileisbiggerthanthevalueof--batch-size.
ImportingCSVData
Arangoimp
359
arangoimpalsooffersthepossibilitytoimportdatafromCSVfiles.ThiscomeshandywhenthedataathandisinCSVformatalreadyandyoudon'twanttospendtimeconvertingthemtoJSONfortheimport.
ToimportdatafromaCSVfile,makesureyourfilecontainstheattributenamesinthefirstrow.Allthefollowinglinesinthefilewillbeinterpretedasdatarecordsandwillbeimported.
TheCSVimportrequiresthedatatohaveahomogeneousstructure.Allrecordsmusthaveexactlythesameamountofcolumnsasthereareheaders.Bydefault,lineswithadifferentnumberofvalueswillnotbeimportedandtherewillbewarningsforthem.Tostillimportlineswithlessvaluesthanintheheader,thereisthe--ignore-missingoption.Ifsettotrue,linesthathaveadifferentamountoffieldswillbeimported.Inthiscaseonlythoseattributeswillbepopulatedforwhichtherearevalues.Attributesforwhichtherearenovaluespresentwillsilentlybediscarded.
Example:
"first","last","age","active","dob"
"John","Connor",25,true
"Jim","O'Brady"
With--ignore-missingthiswillproducethefollowingdocuments:
{"first":"John","last":"Connor","active":true,"age":25}
{"first":"Jim","last":"O'Brady"}
Thecellvaluescanhavedifferentdatatypesthough.Ifacelldoesnothaveanyvalue,itcanbeleftemptyinthefile.Thesevalueswillnotbeimportedsotheattributeswillnot"bethere"indocumentcreated.Valuesenclosedinquoteswillbeimportedasstrings,sotoimportnumericvalues,booleanvaluesorthenullvalue,don'tenclosethevalueinquotesinyourfile.
We'llbeusingthefollowingimportfortheCSVimport:
"first","last","age","active","dob"
"John","Connor",25,true,
"Jim","O'Brady",19,,
"Lisa","Jones",,,"1981-04-09"
Hans,dosSantos,0123,,
Wayne,Brewer,,false,
Thecommandlinetoexecutetheimportis:
>arangoimp--file"data.csv"--typecsv--collection"users"
Theabovedatawillbeimportedinto5documentswhichwilllookasfollows:
{"first":"John","last":"Connor","active":true,"age":25}
{"first":"Jim","last":"O'Brady","age":19}
{"first":"Lisa","last":"Jones","dob":"1981-04-09"}
{"first":"Hans","last":"dosSantos","age":123}
{"first":"Wayne","last":"Brewer","active":false}
Ascanbeseen,valuesleftcompletelyemptyintheinputfilewillbetreatedasabsent.Numericvaluesnotenclosedinquoteswillbetreatedasnumbers.Notethatleadingzerosinnumericvalueswillberemoved.Toimportnumberswithleadingzeros,pleaseusestrings.Theliteralstrueandfalsewillbetreatedasbooleansiftheyarenotenclosedinquotes.Othervaluesnotenclosedinquoteswillbetreatedasstrings.Anyvaluesenclosedinquoteswillbetreatedasstrings,too.
Stringvaluescontainingthequotecharacterortheseparatormustbeenclosedwithquotecharacters.Withinastring,thequotecharacteritselfmustbeescapedwithanotherquotecharacter(orwithabackslashifthe--backslash-escapeoptionisused).
Notethatthequoteandseparatorcharacterscanbeadjustedviathe--quoteand--separatorargumentswheninvokingarangoimp.Thequotecharacterdefaultstothedoublequote(").Tousealiteralquoteinastring,youcanusetwoquotecharacters.Tousebackslashforescapingquotecharacters,pleasesettheoption--backslash-escapetotrue.
TheimportersupportsWindows(CRLF)andUnix(LF)linebreaks.Linebreaksmightalsooccurinsidevaluesthatareenclosedwiththequotecharacter.
Arangoimp
360
Here'sanexampleforusingliteralquotesandnewlinesinsidevalues:
"name","password"
"Foo","r4ndom""123!"
"Bar","wow!
thisisa
multinepassword!"
"Bartholomew""Bart""Simpson","Milhouse"
Extrawhitespaceattheendofeachlinewillbeignored.Whitespaceatthestartoflinesorbetweenfieldvalueswillnotbeignored,sopleasemakesurethatthereisnoextrawhitespaceinfrontofvaluesorbetweenthem.
ImportingTSVData
Youmayalsoimporttab-separatedvalues(TSV)fromafile.Thisformatisverysimple:everylineinthefilerepresentsadatarecord.Thereisnoquotingorescaping.Thatalsomeansthattheseparatorcharacter(whichdefaultstothetabstopsymbol)mustnotbeusedanywhereintheactualdata.
AswithCSV,thefirstlineintheTSVfilemustcontaintheattributenames,andalllinesmusthaveanidenticalnumberofvalues.
Ifadifferentseparatorcharacterorstringshouldbeused,itcanbespecifiedwiththe--separatorargument.
AnexamplecommandlinetoexecutetheTSVimportis:
>arangoimp--file"data.tsv"--typetsv--collection"users"
AttributeNameTranslation
FortheCSVandTSVinputformats,attributenamescanbetranslatedautomatically.ThisisusefulincasetheimportfilehasdifferentattributenamesthanthosethatshouldbeusedinArangoDB.
Acommonusecaseistorenamean"id"columnfromtheinputfileinto"_key"asitisexpectedbyArangoDB.Todothis,specifythefollowingtranslationwheninvokingarangoimp:
>arangoimp--file"data.csv"--typecsv--translate"id=_key"
Othercommoncasesaretorenamecolumnsintheinputfileto_fromand_to:
>arangoimp--file"data.csv"--typecsv--translate"from=_from"--translate"to=_to"
Thetranslateoptioncanbespecifiedmultipletypes.Thesourceattributenameandthetargetattributemustbeseparatedwitha=.
IgnoringAttributes
FortheCSVandTSVinputformats,certainattributenamescanbeignoredonimports.InanArangoDBclustertherearecaseswherethiscancomeinhandy,whenyourdocumentsalreadycontaina_keyattributeandyourcollectionhasashardingattributeotherthan_key:Intheclusterthisconfigurationisnotsupported,becauseArangoDBneedstoguaranteetheuniquenessofthe_keyattributeinallshardsofthecollection.
>arangoimp--file"data.csv"--typecsv--remove-attribute"_key"
Thesamethingwouldapplyifyourdatacontainsan_idattribute:
>arangoimp--file"data.csv"--typecsv--remove-attribute"_id"
ImportingintoanEdgeCollection
Arangoimp
361
arangoimpcanalsobeusedtoimportdataintoanexistingedgecollection.Theimportdatamust,foreachedgetoimport,containatleastthe_fromand_toattributes.Theseindicatewhichothertwodocumentstheedgeshouldconnect.Itisnecessarythattheseattributesaresetforallrecords,andpointtovaliddocumentidsinexistingcollections.
Examples
{"_from":"users/1234","_to":"users/4321","desc":"1234isconnectedto4321"}
Note:Theedgecollectionmustalreadyexistwhentheimportisstarted.Usingthe--create-collectionflagwillnotworkbecausearangoimpwillalwaystrytocreatearegulardocumentcollectionifthetargetcollectiondoesnotexist.
Updatingexistingdocuments
Bydefault,arangoimpwilltrytoinsertalldocumentsfromtheimportfileintothespecifiedcollection.Incasetheimportfilecontainsdocumentsthatarealreadypresentinthetargetcollection(matchingisdoneviathe_keyattributes),thenadefaultarangoimprunwillnotimportthesedocumentsandcomplainaboutuniquekeyconstraintviolations.
However,arangoimpcanbeusedtoupdateorreplaceexistingdocumentsincasetheyalreadyexistinthetargetcollection.Itprovidesthecommand-lineoption--on-duplicatetocontrolthebehaviorincaseadocumentisalreadypresentinthedatabase.
Thedefaultvalueof--on-duplicateiserror.Thismeansthatwhentheimportfilecontainsadocumentthatispresentinthetargetcollectionalready,thentryingtore-insertadocumentwiththesame_keyvalueisconsideredanerror,andthedocumentinthedatabasewillnotbemodified.
Otherpossiblevaluesfor--on-duplicateare:
update:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbeupdatedbyarangoimp.updatewillperformapartialupdateoftheexistingdocument,modifyingonlytheattributesthatarepresentintheimportfileandleavingallotherattributesuntouched.
Thevaluesofsystemattributes_id,_key,_rev,_fromand_tocannotbeupdatedorreplacedinexistingdocuments.
replace:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbereplacebyarangoimp.replacewillreplacetheexistingdocumententirely,resultinginadocumentwithonlytheattributesspecifiedintheimportfile.
Thevaluesofsystemattributes_id,_key,_rev,_fromand_tocannotbeupdatedorreplacedinexistingdocuments.
ignore:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbeignoredandnotmodifiedinthetargetcollection.
When--on-duplicateissettoeitherupdateorreplace,arangoimpwillreturnthenumberofdocumentsupdated/replacedintheupdatedreturnvalue.Whensettoanothervalue,thevalueofupdatedwillalwaysbezero.When--on-duplicateissettoignore,arangoimpwillreturnthenumberofignoreddocumentsintheignoredreturnvalue.Whensettoanothervalue,ignoredwillalwaysbezero.
Itispossibletoperformacombinationofinsertsandupdates/replaceswithasinglearangoimprun.When--on-duplicateissettoupdateorreplace,alldocumentspresentintheimportfilewillbeinsertedintothetargetcollectionprovidedtheyarevalidanddonotalreadyexistwiththespecified_key.Documentsthatarealreadypresentinthetargetcollection(identifiedby_keyattribute)willinsteadbeupdated/replaced.
Arangoimpresultoutput
Anarangoimpimportrunwillprintoutthefinalresultsonthecommandline.Itwillshowthe
numberofdocumentscreated(created)numberofdocumentsupdated/replaced(updated/replaced,onlynon-zeroif--on-duplicatewassettoupdateorreplace,seebelow)numberofwarningsorerrorsthatoccurredontheserverside(warnings/errors)numberofignoreddocuments(onlynon-zeroif--on-duplicatewassettoignore).
Example
created:2
warnings/errors:0
updated/replaced:0
ignored:0
Arangoimp
362
ForCSVandTSVimports,thetotalnumberofinputfilelinesreadwillalsobeprinted(linesread).
arangoimpwillalsoprintoutdetailsaboutwarningsanderrorsthathappenedontheserver-side(ifany).
AttributeNamingandSpecialAttributes
AttributeswhosenamesstartwithanunderscorearetreatedinaspecialwaybyArangoDB:
theoptional_keyattributecontainsthedocument'skey.Ifspecified,thevaluemustbeformallyvalid(e.g.mustbeastringandconformtothenamingconventions).Additionally,thekeyvaluemustbeuniquewithinthecollectiontheimportisrunfor._from:whenimportingintoanedgecollection,thisattributecontainstheidofoneofthedocumentsconnectedbytheedge.Thevalueof_frommustbeasyntacticallyvaliddocumentidandthereferredcollectionmustexist._to:whenimportingintoanedgecollection,thisattributecontainstheidoftheotherdocumentconnectedbytheedge.Thevalueof_tomustbeasyntacticallyvaliddocumentidandthereferredcollectionmustexist._rev:thisattributecontainstherevisionnumberofadocument.However,therevisionnumbersaremanagedbyArangoDBandcannotbespecifiedonimport.Thusanyvalueinthisattributeisignoredonimport.
Ifyouimportvaluesinto_key,youshouldmakesuretheyarevalidandunique.
Whenimportingdataintoanedgecollection,youshouldmakesurethatallimportdocumentscan_fromand_toandthattheirvaluespointtoexistingdocuments.
Toavoidspecifyingcompletedocumentids(consistingofcollectionnamesanddocumentkeys)for_fromand_tovalues,therearetheoptions--from-collection-prefixand--to-collection-prefix.Ifspecified,thesevalueswillbeautomaticallyprependedtoeachvaluein_from(or_toresp.).Thisallowsspecifyingonlydocumentkeysinside_fromand/or_to.
Example
>arangoimp--from-collection-prefixusers--to-collection-prefixproducts...
Importingthefollowingdocumentwillthencreateanedgebetweenusers/1234andproducts/4321:
{"_from":"1234","_to":"4321","desc":"users/1234isconnectedtoproducts/4321"}
Arangoimp
363
DumpingDatafromanArangoDBdatabaseTodumpdatafromanArangoDBserverinstance,youwillneedtoinvokearangodump.Dumpscanbere-importedwitharangorestore.arangodumpcanbeinvokedbyexecutingthefollowingcommand:
unix>arangodump--output-directory"dump"
ThiswillconnecttoanArangoDBserveranddumpallnon-systemcollectionsfromthedefaultdatabase(_system)intoanoutputdirectorynameddump.Invokingarangodumpwillfailiftheoutputdirectoryalreadyexists.Thisisanintentionalsecuritymeasuretopreventyoufromaccidentallyoverwritingalreadydumpeddata.Ifyouarepositivethatyouwanttooverwritedataintheoutputdirectory,youcanusetheparameter--overwritetruetoconfirmthis:
unix>arangodump--output-directory"dump"--overwritetrue
arangodumpwillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabaseoradifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:
--server.database:nameofthedatabasetoconnectto--server.endpoint:endpointtoconnectto--server.username:username--server.password:passwordtouse(omitthisandyou'llbepromptedforthepassword)--server.authentication:whetherornottouseauthentication
Here'sanexampleofdumpingdatafromanon-standardendpoint,usingadedicateddatabasename:
unix>arangodump--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--output-directory
"dump"
Whenfinished,arangodumpwillprintoutasummarylinewithsomeaggregatestatisticsaboutwhatitdid,e.g.:
Processed43collection(s),wrote408173500byte(s)intodatafiles,sent88batch(es)
Bydefault,arangodumpwilldumpbothstructuralinformationanddocumentsfromallnon-systemcollections.Toadjustthis,therearethefollowingcommand-linearguments:
--dump-data:settotruetoincludedocumentsinthedump.Settofalsetoexcludedocuments.Thedefaultvalueistrue.--include-system-collections:whetherornottoincludesystemcollectionsinthedump.Thedefaultvalueisfalse.
Forexample,toonlydumpstructuralinformationofallcollections(includingsystemcollections),use:
unix>arangodump--dump-datafalse--include-system-collectionstrue--output-directory"dump"
Torestrictthedumptojustspecificcollections,thereisisthe--collectionoption.Itcanbespecifiedmultipletimesifrequired:
unix>arangodump--collectionmyusers--collectionmyvalues--output-directory"dump"
Structuralinformationforacollectionwillbesavedinfileswithnamepattern.structure.json.EachstructurefilewillcontainsaJSONobjectwiththeseattributes:
parameters:containsthecollectionpropertiesindexes:containsthecollectionindexes
Documentdataforacollectionwillbesavedinfileswithnamepattern.data.json.Eachlineinadatafileisadocumentinsertion/updateordeletionmarker,alongsidewithsomemetadata.
StartingwithVersion2.1ofArangoDB,thearangodumptoolalsosupportssharding.Simplypointittooneofthecoordinatorsanditwillbehaveexactlyasdescribedabove,workingonshardedcollectionsinthecluster.
Arangodump
364
However,asopposedtothesingleinstancesituation,thisoperationdoesnotguaranteetodumpaconsistentsnapshotifwriteoperationshappenduringthedumpoperation.Itisthereforerecommendednottoperformanydata-modifcationoperationsontheclusterwhilstarangodumpisrunning.
Asabove,theoutputwillbeonestructuredescriptionfileandonedatafilepershardedcollection.Notethatthedatainthedatafileissortedfirstbyshardsandwithineachshardbyascendingtimestamp.Thestructuralinformationofthecollectioncontainsthenumberofshardsandtheshardkeys.
NotethattheversionofthearangodumpclienttoolneedstomatchtheversionoftheArangoDBserveritconnectsto.
Advancedclusteroptions
Startingwithversion3.1.17,collectionsmaybecreatedwithsharddistributionidenticaltoanexistingprototypicalcollection;i.e.shardsaredistributedintheverysamepatternasintheprototypecollection.Suchcollectionscannotbedumpedwithoutthereferencecollectionorarangodumpwithyieldanerror.
unix>arangodump--collectionclonedCollection--output-directory"dump"
ERRORCollectionclonedCollection'ssharddistributionisbasedonathatofcollectionprototypeCollection,whichisnotdumpe
dalong.Youmaydumpthecollectionregardlessofthemissingprototypecollectionbyusingthe--ignore-distribute-shards-lik
e-errorsparameter.
Therearetwowaystoapproachthatproblem:Solveit,i.e.dumptheprototypecollectionalong:
unix>arangodump--collectionclonedCollection--collectionprototypeCollection--output-directory"dump"
Processed2collection(s),wrote81920byte(s)intodatafiles,sent1batch(es)
Oroverridethatbehaviourtobeabletodumpthecollectionindividually.
unix>arangodump--collectionBclonedCollection--output-directory"dump"--ignore-distribute-shards-like-errors
Processed1collection(s),wrote34217byte(s)intodatafiles,sent1batch(es)
Nothatinconsequence,restoringsuchacollectionwithoutitsprototypeisaffected.arangorestore
Encryption
IntheArangoDBEnterpriseEditiontherearetheadditionalparameters:
Encryptionkeystoredinfile
--encryption.keyfilepath-of-keyfile
Thefilepath-to-keyfilemustcontaintheencryptionkey.Thisfilemustbesecured,sothatonlyarangodcanaccessit.Youshouldalsoensurethatincasesome-onestealsthehardware,hewillnotbeabletoreadthefile.Forexample,byencryption/mytmpfsorcreatingain-memoryfile-systemunder/mytmpfs.
Encryptionkeygeneratedbyaprogram
--encryption.key-generatorpath-to-my-generator
Theprogrampath-to-my-generatormustoutputtheencryptiononstandardoutputandexit.
Creatingkeys
Theencryptionkeyfilemustcontain32bytesofrandomdata.
Youcancreateitwithacommandlinethis.
ddif=/dev/randombs=1count=32of=yourSecretKeyFile
Arangodump
365
Forsecurity,itisbesttocreatethesekeysoffline(awayfromyourdatabaseservers)anddirectlystoretheminyousecretmanagementtool.
Arangodump
366
ArangorestoreToreloaddatafromadumppreviouslycreatedwitharangodump,ArangoDBprovidesthearangorestoretool.
Pleasenotethatarangorestoremustnotbeusedtocreateseveralsimilardatabaseinstancesinoneinstallation.
Thismeansifyouhaveanarangodumpoutputofdatabasea,andyoucreateaseconddatabasebonthesameinstanceofArangoDB,andrestorethedumpofaintob-dataintegritycannotbeguaranteed.
ReloadingDataintoanArangoDBdatabase
Invokingarangorestore
arangorestorecanbeinvokedfromthecommand-lineasfollows:
unix>arangorestore--input-directory"dump"
ThiswillconnecttoanArangoDBserverandreloadstructuralinformationanddocumentsfoundintheinputdirectorydump.Pleasenotethattheinputdirectorymusthavebeencreatedbyrunningarangodumpbefore.
arangorestorewillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabaseoradifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:
--server.database:nameofthedatabasetoconnectto--server.endpoint:endpointtoconnectto--server.username:username--server.password:passwordtouse(omitthisandyou'llbepromptedforthepassword)--server.authentication:whetherornottouseauthentication
Sinceversion2.6arangorestoreprovidestheoption--create-database.Settingthisoptiontotruewillcreatethetargetdatabaseifitdoesnotexist.Whencreatingthetargetdatabase,theusernameandpasswordspassedtoarangorestore(inoptions--server.usernameand--server.password)willbeusedtocreateaninitialuserforthenewdatabase.
Theoption--force-same-databaseallowsrestrictingarangorestoreoperationstoadatabasewiththesamenameasinthesourcedump's"dump.json"file.Itcanthusbeusedtopreventrestoringdataintoa"wrong"databasebyaccident.
Forexample,ifadumpwastakenfromdatabasea,andtherestoreisattemptedintodatabaseb,thenwiththe--force-same-databaseoptionsettotrue,arangorestorewillabortinstantly.
The--force-same-databaseoptionissettofalsebydefaulttoensurebackwards-compatibility.
Here'sanexampleofreloadingdatatoanon-standardendpoint,usingadedicateddatabasename:
unix>arangorestore--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--input-directo
ry"dump"
Tocreatethetargetdatabasewherestoring,useacommandlikethis:
unix>arangorestore--server.usernamebackup--server.databasenewdb--create-databasetrue--input-directory"dump"
arangorestorewillprintoutitsprogresswhilerunning,andwillendwithalineshowingsomeaggregatestatistics:
Processed2collection(s),read2256byte(s)fromdatafiles,sent2batch(es)
Bydefault,arangorestorewillre-createallnon-systemcollectionsfoundintheinputdirectoryandloaddataintothem.Ifthetargetdatabasealreadycontainscollectionswhicharealsopresentintheinputdirectory,theexistingcollectionsinthedatabasewillbedroppedandre-createdwiththedatafoundintheinputdirectory.
Arangorestore
367
Thefollowingparametersareavailabletoadjustthisbehavior:
--create-collection:settotruetocreatecollectionsinthetargetdatabase.Ifthetargetdatabasealreadycontainsacollectionwiththesamename,itwillbedroppedfirstandthenre-createdwiththepropertiesfoundintheinputdirectory.Settofalsetokeepexistingcollectionsinthetargetdatabase.Ifsettofalseandarangorestoreencountersacollectionthatispresentinboththetargetdatabaseandtheinputdirectory,itwillabort.Thedefaultvalueistrue.--import-data:settotruetoloaddocumentdataintothecollectionsinthetargetdatabase.Settofalsetonotloadanydocumentdata.Thedefaultvalueistrue.--include-system-collections:whetherornottoincludesystemcollectionswhenre-creatingcollectionsorreloadingdata.Thedefaultvalueisfalse.
Forexample,to(re-)createallnon-systemcollectionsandloaddocumentdataintothem,use:
unix>arangorestore--create-collectiontrue--import-datatrue--input-directory"dump"
Thiswilldroppotentiallyexistingcollectionsinthetargetdatabasethatarealsopresentintheinputdirectory.
Toincludesystemcollectionstoo,use--include-system-collectionstrue:
unix>arangorestore--create-collectiontrue--import-datatrue--include-system-collectionstrue--input-directory"dump"
To(re-)createallnon-systemcollectionswithoutloadingdocumentdata,use:
unix>arangorestore--create-collectiontrue--import-datafalse--input-directory"dump"
Thiswillalsodropexistingcollectionsinthetargetdatabasethatarealsopresentintheinputdirectory.
Tojustloaddocumentdataintoallnon-systemcollections,use:
unix>arangorestore--create-collectionfalse--import-datatrue--input-directory"dump"
Torestrictreloadingtojustspecificcollections,thereisisthe--collectionoption.Itcanbespecifiedmultipletimesifrequired:
unix>arangorestore--collectionmyusers--collectionmyvalues--input-directory"dump"
Collectionswillbeprocessedbyinalphabeticalorderbyarangorestore,withalldocumentcollectionsbeingprocessedbeforealledgecollections.Thisistoensurethatreloadingdataintoedgecollectionswillhavethedocumentcollectionslinkedinedges(_fromand_toattributes)loaded.
Encryption
Seearangodumpfordetails.
RestoringRevisionIdsandCollectionIds
arangorestorewillreloaddocumentandedgesdatawiththeexactsame_key,_fromand_tovaluesfoundintheinputdirectory.However,whenloadingdocumentdata,itwillassignitsownvaluesforthe_revattributeofthereloadeddocuments.Thoughthisdifferenceisintentional(normally,everyservershouldcreateitsown_revvalues)theremightbesituationswhenitisrequiredtore-usetheexactsame_revvaluesforthereloadeddata.Thiscanbeachievedbysettingthe--recycle-idsparametertotrue:
unix>arangorestore--collectionmyusers--collectionmyvalues--input-directory"dump"
Notethatsetting--recycle-idstotruewillalsocausecollectionstobe(re-)createdinthetargetdatabasewiththeexactsamecollectionidasintheinputdirectory.Anypotentiallyexistingcollectioninthetargetdatabasewiththesamecollectionidwillthenbedropped.
ReloadingDataintoadifferentCollection
Arangorestore
368
Withsomecreativityyoucanusearangodumpandarangorestoretotransferdatafromonecollectionintoanother(eitheronthesameserverornot).Forexample,tocopydatafromacollectionmyvaluesindatabasemydbintoacollectionmycopyvaluesindatabasemycopy,youcanstartwiththefollowingcommand:
unix>arangodump--collectionmyvalues--server.databasemydb--output-directory"dump"
Thiswillcreatetwofiles,myvalues.structure.jsonandmyvalues.data.json,intheoutputdirectory.Toloaddatafromthedatafileintoanexistingcollectionmycopyvaluesindatabasemycopy,renamethefilestomycopyvalues.structure.jsonandmycopyvalues.data.json.Afterthat,runthefollowingcommand:
unix>arangorestore--collectionmycopyvalues--server.databasemycopy--input-directory"dump"
Usingarangorestorewithsharding
AsofVersion2.1thearangorestoretoolsupportssharding.Simplypointittooneofthecoordinatorsinyourclusteranditwillworkasusualbutonshardedcollectionsinthecluster.
Ifarangorestoreisaskedtodropandre-createacollection,itwillusethesamenumberofshardsandthesameshardkeysaswhenthecollectionwasdumped.Thedistributionoftheshardstotheserverswillalsobethesameasatthetimeofthedump.ThismeansinparticularthatDBserverswiththesameIDsasbeforemustbepresentintheclusterattimeoftherestore.
Ifacollectionwasdumpedfromasingleinstance,onecanmanuallyaddthestructuraldescriptionfortheshardkeysandthenumberanddistributionoftheshardsandthentherestoreintoaclusterwillwork.
IfyourestoreacollectionthatwasdumpedfromaclusterintoasingleArangoDBinstance,thenumberofshardsandtheshardkeyswillsilentlybeignored.
Notethatinacluster,everynewlycreatedcollectionwillhaveanewID,itisnotpossibletoreusetheIDfromtheoriginallydumpedcollection.ThisisforsafetyreasonstoensureconsistencyofIDs.
Restoringcollectionswithshardingprototypes
arangorestorewillyieldanerror,whiletryingtorestoreacollection,whosesharddistributionfollowsacollection,whichdoesnotexistintheclusterandwhichwasnotdumpedalong:
unix>arangorestore--collectionclonedCollection--server.databasemydb--input-directory"dump"
ERRORgoterrorfromserver:HTTP500(InternalServerError):ArangoError1486:mustnothaveadistributeShardsLikeattribute
pointingtoanunknowncollection
Processed0collection(s),read0byte(s)fromdatafiles,sent0batch(es)
Thecollectioncanberestoredbyoverridingtheerrormessageasfollows:
unix>arangorestore--collectionclonedCollection--server.databasemydb--input-directory"dump"--ignore-distribute-shards-li
ke-errors
RestoreintoanauthenticationenabledArangoDB
OfcourseyoucanrestoredataintoapasswordprotectedArangoDBaswell.Howeverthisrequirescertainuserrightsfortheuserusedintherestoreprocess.TherightsaredescribedindetailintheManagingUserschapter.Forrestorethisshortoverviewissufficient:
Whenimportingintoanexistingdatabase,thegivenuserneedsAdministrateaccessonthisdatabase.WhencreatinganewDatabaseduringrestore,thegivenuserneedsAdministrateaccesson_system.TheuserwillbepromotedwithAdministrateaccessonthenewlycreateddatabase.
Arangorestore
369
ExportingDatafromanArangoDBdatabaseToexportdatafromanArangoDBserverinstance,youwillneedtoinvokearangoexport.arangoexportcanbeinvokedbyexecutingthefollowingcommand:
unix>arangoexport--collectiontest--output-directory"dump"
Thisexportsthecollectionstestintothedirectorydumpasonebigjsonarray.Everyentryinthisarrayisonedocumentfromthecollectionwithoutaspecificorder.Toexportmorethanonecollectionatatimespecifymultiple--collectionoptions.
Thedefaultoutputdirectoryisexport.
arangoexportwillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabaseoradifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:
--server.database:nameofthedatabasetoconnectto--server.endpoint:endpointtoconnectto--server.username:username--server.password:passwordtouse(omitthisandyou'llbepromptedforthepassword)--server.authentication:whetherornottouseauthentication
Here'sanexampleofexportingdatafromanon-standardendpoint,usingadedicateddatabasename:
unix>arangoexport--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--collectiontes
t--output-directory"my-export"
Whenfinished,arangoexportwillprintoutasummarylinewithsomeaggregatestatisticsaboutwhatitdid,e.g.:
Processed2collection(s),wrote9031763Byte(s),78HTTPrequest(s)
ExportJSON
unix>arangoexport--typejson--collectiontest
Thisexportsthecollectiontestintotheoutputdirectoryexportasonejsonarray.Everyarrayentryisonedocumentfromthecollectiontest
ExportJSONL
unix>arangoexport--typejsonl--collectiontest
Thisexportsthecollectiontestintotheoutputdirectoryexportasjsonl.Everylineintheexportisonedocumentfromthecollectiontestasjson.
ExportCSV
unix>arangoexport--typecsv--collectiontest--fields_key,_id,_rev
ThisexportsthecollectiontestintotheoutputdirectoryexportasCSV.Thefirstlinecontainstheheaderwithallfieldnames.EachlineisonedocumentrepresentedasCSVandseparatedwithacomma.ObjectsandArraysarerepresentedasaJSONstring.
Arangoexport
370
ExportXML
unix>arangoexport--typexml--collectiontest
ThisexportsthecollectiontestintotheoutputdirectoryexportasgenericXML.TherootelementofthegeneratedXMLfileisnamedcollection.EachdocumentinthecollectionisexportedinadocXMLattribute.Eachdocumentattributeisexportinagenericattelement,whichhasatypeattributeindicatingtheattributevalue,andavalueattributecontainingtheattribute'svalue.
ExportXGMMLXGMMLisanXMLapplicationbasedonGML.ToviewtheXGMMLfileyoucanuseforexampleCytoscape.
importantnoteIfyouexportallattributes(--xgmml-label-onlyfalse)keepinmindthataatrributenamestypehavetobethesametypeforalldocuments.Itwontworkifyouhaveaattributenamedrankthatisinonedocumentastringandinanotherdocumentainteger.
Bad
//doc1
{
"rank":1
}
//doc2
{
"rank":"2"
}
Good
//doc1
{
"rank":1
}
//doc2
{
"rank":2
}
XGMMLspecificoptions--xgmml-label-attributespecifythenameoftheattributethatwillbecomethelabelinthexgmmlfile.
--xgmml-label-onlysettotruewillonlyexportthelabelwithoutanyattributesinedgesornodes.
exportbasedoncollections
unix>arangoexport--typexgmml--graph-namemygraph--collectionvertex--collectionedge
Thisexportstheaunnamedgraphwithvertexcollectionvertexandedgecollectionedgeintothexgmmlfilemygraph.xgmml.
exportbasedonanamedgraph
unix>arangoexport--typexgmml--graph-namemygraph
Arangoexport
371
Thisexportsthenamedgraphmygraphintothexgmmlfilemygraph.xgmml.
exportXGMMLwithoutattributes
unix>arangoexport--typexgmml--graph-namemygraph--xgmml-label-onlytrue
Thisexportsthenamedgraphmygraphintothexgmmlfilemygraph.xgmmlwithoutthe<att>taginnodesandedges.
exportXGMMLwithaspecificlabel
unix>arangoexport--typexgmml--graph-namemygraph--xgmml-label-attributename
Thisexportsthenamedgraphmygraphintothexgmmlfilemygraph.xgmmlwithalabelfromdocumentsattributenameinsteadofthedefaultattributelabel.
ExportviaAQLquery
unix>arangoexport--typejsonl--query"forbookinbooksfilterbook.sells>100returnbook"
Exportviaanaqlqueryallowsyoutoexportthereturneddataasthetypespecifiedwith--type.Theexampleexportsallbooksasjsonlthataresoldmorethan100times.
Arangoexport
372
ManagingUsersTheusermanagementinArangoDB3issimilartotheonesfoundinMySQL,PostgreSQL,orotherdatabasesystems.
Usermanagementispossibleinthewebinterfaceandinarangoshwhileloggedontothe_systemdatabase.
Notethattheonlyusernamesmustnotstartwith:role:.
ActionsandAccessLevelsAnArangoDBservercontainsalistofusers.Italsodefinesvariousaccesslevelsthatcanbeassignedtoauser(fordetails,seebelow)andthatareneededtoperformcertainactions.Theseactionscanbegroupedintothreecategories:
serveractionsdatabaseactionscollectionactions
Theserveractionsare
createuser:allowstocreateanewuser.
updateuser:allowstochangetheaccesslevelsanddetailsofanexistinguser.
dropuser:allowstodeleteanexistinguser.
createdatabase:allowstocreateanewdatabase.
dropdatabase:allowstodeleteanexistingdatabase.
shutdownserver:removeserverfromclusterandshutdown
Thedatabaseactionsaretiedtoagivendatabase,andaccesslevelsmustbesetforeachdatabaseindividually.Foragivendatabasetheactionsare
createcollection:allowstocreateanewcollectioninthegivendatabase.
updatecollection:allowstoupdatepropertiesofanexistingcollection.
dropcollection:allowstodeleteanexistingcollection.
createindex:allowstocreateanindexforanexistingcollectioninthegivendatabase.
dropindex:allowstodeleteanindexofanexistingcollectioninthegivendatabase.
Thecollectionactionsaretiedtoagivencollectionofagivendatabase,andaccesslevelsmustbesetforeachcollectionindividually.Foragivencollectiontheactionsare
readdocument:readadocumentofthegivencollection.
createdocument:createsanewdocumentinthegivencollection.
modifydocument:modifiesanexistingdocumentofthegivencollection,thiscanbeanupdateorreplaceoperation.
dropdocument:deletesanexistingdocumentofthegivencollection.
truncatecollection:deletesalldocumentsofagivencollection.
Toperformactionsontheserverleveltheuserneedsatleastthefollowingaccesslevels.TheaccesslevelsareAdministrateandNoaccess:
serveraction serverlevel
createadatabase Administrate
dropadatabase Administrate
createauser Administrate
ManagingUsers
373
updateauser Administrate
updateuseraccesslevel Administrate
dropauser Administrate
shutdownserver Administrate
Toperformactionsinaspecificdatabase(likecreatingordroppingcollections),auserneedsatleastthefollowingaccesslevel.ThepossibleaccesslevelsfordatabasesareAdministrate,AccessandNoaccess.TheaccesslevelsforcollectionsareRead/Write,ReadOnlyandNoAccess.
databaseaction databaselevel collectionlevel
createcollection Administrate Read/Write
listcollections Access ReadOnly
renamecollection Administrate Read/Write
modifycollectionproperties Administrate Read/Write
readproperties Access ReadOnly
dropcollection Administrate Read/Write
createanindex Administrate Read/Write
dropanindex Administrate Read/Write
seeindexdefinition Access ReadOnly
NotethattheaccesslevelAccessforadatabaseisalwaysrequiredtoperformanyactiononacollectioninthatdatabase.
Forcollectionsauserneedsthefollowingaccesslevelstothegivendatabaseandthegivencollection.TheaccesslevelsforthedatabaseareAdministrate,AccessandNoaccess.TheaccesslevelsforthecollectionareRead/Write,ReadOnlyandNoAccess.
action collectionlevel databaselevel
readadocument Read/WriteorReadOnly AdministrateorAccess
createadocument Read/Write AdministrateorAccess
modifyadocument Read/Write AdministrateorAccess
dropadocument Read/Write AdministrateorAccess
truncateacollection Read/Write AdministrateorAccess
Example
Forexample,given
adatabaseexampleacollectiondatainthedatabaseexampleauserJohnSmith
IftheuserJohnSmithisassignedtheaccesslevelAccessforthedatabaseexampleandthelevelRead/Writeforthecollectiondata,thentheuserisallowedtoread,create,modifyordeletedocumentsinthecollectiondata.Buttheuseris,forexample,notallowedtocreateindexesforthecollectiondatanorcreatenewcollectionsinthedatabaseexample.
GrantingAccessLevels
Accesslevelscanbemanagedviathewebinterfaceorinarangosh.
Inordertograntanaccessleveltoauser,youcanassignoneofthreeaccesslevelsforeachdatabaseandoneofthreelevelsforeachcollectioninadatabase.Theserveraccesslevelfortheuserfollowsfromthedatabaseaccesslevelinthe_systemdatabase,itisAdministrateifandonlyifthedatabaseaccesslevelisAdministrate.NotethatthismeansthatdatabaseaccesslevelAccessdoesnotgrantauserserveraccesslevelAdministrate.
ManagingUsers
374
InitialAccessLevels
WhenausercreatesadatabasetheaccessleveloftheuserforthatdatabaseissettoAdministrate.Thesameistrueforcreatingacollection,inthiscasetheusergetRead/Writeaccesstothecollection.
WildcardDatabaseAccessLevel
Withtheabovedefinition,onemustdefinethedatabaseaccesslevelforalldatabase/userpairsintheserver,whichwouldbeverytedious.Inordertosimplifythisprocess,itispossibletodefine,forauser,awildcarddatabaseaccesslevel.Thiswildcardisusedifthedatabaseaccesslevelisnotexplicitlydefinedforacertaindatabase.EachnewcreateduserhasaninitialdatabasewildcardofNoAccess.
Changingthewildcarddatabaseaccesslevelforauserwillchangetheaccesslevelforalldatabasesthathavenoexplicitlydefinedaccesslevel.Notethatthisincludesdatabaseswhichwillbecreatedinthefutureandforwhichnoexplicitaccesslevelsaresetforthatuser!
Ifyoudeletethewildcard,thedefaultaccesslevelisdefinedasNoAccess.
TherootuserhasaninitialdatabasewildcardofAdministrate.
Example
AssumeuserJohnSmithhasthefollowingdatabaseaccesslevels:
accesslevel
database* Access
databaseshop1 Administrate
databaseshop2 NoAccess
ThiswillgivetheuserJohnSmiththefollowingdatabaselevelaccess:
databaseshop1:Administratedatabaseshop2:NoAccessdatabasesomething:Access
Ifthewildcard*ischangedfromAccesstoNoAccessthenthepermissionswillchangeasfollows:
databaseshop1:Administratedatabaseshop2:NoAccessdatabasesomething:NoAccess
WildcardCollectionAccessLevel
Foreachuseranddatabasethereisawildcardcollectionaccesslevel.Thislevelisusedforallcollectionspairswithoutanexplicitlydefinedcollectionaccesslevel.Notethatthisincludescollectionswhichwillbecreatedinthefutureandforwhichnoexplicitaccesslevelsaresetforathatuser!EachnewcreateduserhasaninitialcollectionwildcardofNoAccess.
Ifyoudeletethewildcard,thesystemdefaultstoNoAccess.
TherootuserhasaninitialcollectionwildcardofRead/Writeineverydatabase.
Whencreatingauserthroughdb._createDatabase(name,options,users)theaccessleveloftheuserforthisdatabasewillbesettoAdministrateandthewildcardforallcollectionswithinthisdatabasewillbesettoRead/Write.
Example
AssumeuserJohnSmithhasthefollowingdatabaseaccesslevels:
accesslevel
database* Access
andcollectionaccesslevels:
accesslevel
ManagingUsers
375
database*,collection* Read/Write
databaseshop1,collectionproducts Read-Only
databaseshop1,collection* NoAccess
databaseshop2,collection* Read-Only
Thentheuserdoewillgetthefollowingcollectionaccesslevels:
databaseshop1,collectionproducts:Read-Onlydatabaseshop1,collectioncustomers:NoAccessdatabaseshop2,collectionreviews:Read-Onlydatabasesomething,collectionelse:Read/Write
Explanation:
Databaseshop1,collectionproductsdirectlymatchesadefinedaccesslevel.ThislevelisdefinedasRead-Only.
Databaseshop1,collectioncustomersdoesnotmatchadefinedaccesslevel.However,databaseshop1matchesandthewildcardinthisdatabaseforcollectionlevelisNoAccess.
Databaseshop2,collectionreviewsdoesnotmatchadefinedaccesslevel.However,databaseshop2matchesandthewildcardinthisdatabaseforcollectionlevelisRead-Only.
Databasesomehing,collectionelsedoesnotmatchadefinedaccesslevel.Thedatabasesomethingalsodoeshaveadirectmatches.Thereforethewildcardisselected.ThelevelisRead/Write.
SystemCollections
Theaccesslevelforsystemcollectionscannotbechanged.Theyfollowdifferentrulesthanuserdefinedcollectionsandmaychangewithoutfurthernotice.Currentlythesystemcollectionsfollowtheserules:
collection accesslevel
_users(in_system) NoAccess
_queues Read-Only
_frontend Read/Write
* sameasdb
AllothersystemcollectionshaveaccesslevelRead/WriteiftheuserhasAdministrateaccesstothedatabase.TheyhaveaccesslevelRead/OnlyiftheuserhasAccesstothedatabase.
TomodifythesesystemcollectionsyoushouldalwaysusethespecializedAPIsprovidedbyArangoDB.Forexamplenouserhasaccesstothe_userscollectioninthe_systemdatabase.Allchangestotheaccesslevelsmustbedoneusingthe@arangodb/usersmodule,the/_users/APIorthewebinterface.
ManagingUsers
376
ManagingUsersintheArangoDBShellPleasenote,thatforbackwardcompatibilitytheserveraccesslevelsfollowfromthedatabaseaccesslevelonthedatabase_system.
Alsonotethattheserveranddatabaseaccesslevelsarerepresentedas
rw:forAdministratero:forAccessnone:forNoaccess
Thisisagainforbackwardcompatibility.
Example
Fireuparangoshandrequiretheusersmodule.Useittocreateanewuser:
arangosh>varusers=require('@arangodb/users');
arangosh>users.save('JohnSmith','mypassword');
CreatesausercalledJohnSmith.Thisuserwillhavenoaccessatall.
arangosh>users.grantDatabase('JohnSmith','testdb','rw');
ThisgrantstheuserAdministrateaccesstothedatabasetestdb.revokeDatabasewillrevokethisaccesslevelsetting.
Note:Beawarethatfrom3.2onwardsthegrantDatabasewillnotautomaticallygrantuserstheaccessleveltowriteorreadcollectionsinadatabase.IfyougrantaccesstoadatabasetestdbyouwilladditionallyneedtoexplicitlygrantaccesslevelstoindividualcollectionsviagrantCollection.
Theupgradeprocedurefrom3.1to3.2setsthewildcarddatabaseaccesslevelforalluserstoAdministrateandsetsthewildcardcollectionaccesslevelforalluser/databasepairstoRead/Write.
arangosh>users.grantCollection('JohnSmith','testdb','testcoll','rw');
Saveusers.save(user,passwd,active,extra)
ThiswillcreateanewArangoDBuser.Theusernamemustbespecifiedinuserandmustnotbeempty.
Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.IfyoupassthespecialvalueARANGODB_DEFAULT_ROOT_PASSWORD,thepasswordwillbesetthevaluestoredintheenvironmentvariableARANGODB_DEFAULT_ROOT_PASSWORD.ThiscanbeusedtopassaninstancevariableintoArangoDB.Forexample,theinstanceidentifierfromAmazon.
Iftheactiveattributeisnotspecified,itdefaultstotrue.Theextraattributecanbeusedtosavecustomdatawiththeuser.
Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,ortherealreadyexistsauserwiththespecifiedname.
Note:Theuserwillnothavepermissiontoaccessanydatabase.YouneedtogranttheaccessrightsforoneormoredatabasesusinggrantDatabase.
Examples
arangosh>require('@arangodb/users').save('my-user','my-secret-password');
showexecutionresults
InArangosh
377
GrantDatabaseusers.grantDatabase(user,database,type)
Thisgrantstype('rw','ro'or'none')accesstothedatabasefortheuser.Ifdatabaseis"*",thissetsthewildcarddatabaseaccesslevelfortheuseruser.
Theserveraccesslevelfollowsfromtheaccesslevelforthedatabase_system.
RevokeDatabaseusers.revokeDatabase(user,database)
Thisclearstheaccesslevelsettingtothedatabasefortheuserandthewildcarddatabaseaccesssettingforthisuserkicksin.IncasenowildcardaccesswasdefinedthedefaultisNoAccess.Thiswillalsocleartheaccesslevelsforallthecollectionsinthisdatabase.
GrantCollectionusers.grantCollection(user,database,collection,type)
Thisgrantstype('rw','ro'or'none')accessleveltothecollectionindatabasefortheuser.Ifcollectionis"*"thissetsthewildcardcollectionaccesslevelfortheuseruserindatabasedatabase.
RevokeCollectionusers.revokeCollection(user,database)
Thisclearstheaccesslevelsettingtothecollectioncollectionfortheuseruser.ThesystemwilleitherfallbacktothewildcardcollectionaccesslevelordefaulttoNoAccess
Replaceusers.replace(user,passwd,active,extra)
ThiswilllookupanexistingArangoDBuserandreplaceitsuserdata.
Theusernamemustbespecifiedinuser,andauserwiththespecifiednamemustalreadyexistinthedatabase.
Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.
Iftheactiveattributeisnotspecified,itdefaultstotrue.Theextraattributecanbeusedtosavecustomdatawiththeuser.
Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,orifthespecifiedusercannotbefoundinthedatabase.
Note:thisfunctionwillnotworkfromwithinthewebinterface
Examples
arangosh>require("@arangodb/users").replace("my-user","my-changed-password");
showexecutionresults
Updateusers.update(user,passwd,active,extra)
ThiswillupdateanexistingArangoDBuserwithanewpasswordandotherdata.
Theusernamemustbespecifiedinuserandtheusermustalreadyexistinthedatabase.
InArangosh
378
Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.
Iftheactiveattributeisnotspecified,thecurrentvaluesavedfortheuserwillnotbechanged.Thesameistruefortheextraattribute.
Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,orifthespecifiedusercannotbefoundinthedatabase.
Examples
arangosh>require("@arangodb/users").update("my-user","my-secret-password");
showexecutionresults
isValidusers.isValid(user,password)
Checkswhetherthegivencombinationofusernameandpasswordisvalid.Thefunctionwillreturnabooleanvalueifthecombinationofusernameandpasswordisvalid.
Eachcalltothisfunctionispenalizedbytheserversleepingarandomamountoftime.
Examples
arangosh>require("@arangodb/users").isValid("my-user","my-secret-password");
true
Removeusers.remove(user)
RemovesanexistingArangoDBuserfromthedatabase.
TheusernamemustbespecifiedinUserandthespecifiedusermustexistinthedatabase.
Thismethodwillfailiftheusercannotbefoundinthedatabase.
Examples
arangosh>require("@arangodb/users").remove("my-user");
Documentusers.document(user)
FetchesanexistingArangoDBuserfromthedatabase.
Theusernamemustbespecifiedinuser.
Thismethodwillfailiftheusercannotbefoundinthedatabase.
Examples
arangosh>require("@arangodb/users").document("my-user");
showexecutionresults
All
InArangosh
379
users.all()
FetchesallexistingArangoDBusersfromthedatabase.
Examples
arangosh>require("@arangodb/users").all();
showexecutionresults
Reloadusers.reload()
Reloadstheuserauthenticationdataontheserver
Alluserauthenticationdataisloadedbytheserveronceonstartuponlyandiscachedafterthat.Whenusersgetaddedordeleted,acacheflushisdoneautomatically,andthiscanbeperformedbyacalltothismethod.
Examples
arangosh>require("@arangodb/users").reload();
Permissionusers.permission(user,database[,collection])
Fetchestheaccessleveltothedatabaseoracollection.
Theuseranddatabasenamemustbespecified,optionallyyoucanspecifythecollectionname.
Thismethodwillfailiftheusercannotbefoundinthedatabase.
Examples
arangosh>require("@arangodb/users").permission("my-user","testdb");
rw
InArangosh
380
Command-lineoptions
GeneralOptions
Generalhelp--help
-h
Printsalistofthemostcommonoptionsavailableandthenexits.Inordertoseealloptionsuse--help-all.
Version--version
-v
Printstheversionoftheserverandexits.
ConfigurationFiles
Optionscanbespecifiedonthecommandlineorinconfigurationfiles.IfastringVariableoccursinthevalue,itisreplacedbythecorrespondingenvironmentvariable.
--configurationfilename
-cfilename
Specifiesthenameoftheconfigurationfiletouse.
Ifthiscommandisnotpassedtotheserver,thenbydefault,theserverwillattempttofirstlocateafilenamed~/.arango/arangod.confintheuser'shomedirectory.
Ifnosuchfileisfound,theserverwillproceedtolookforafilearangod.confinthesystemconfigurationdirectory.Thesystemconfigurationdirectoryisplatform-specific,andmaybechangedwhencompilingArangoDByourself.Itmaydefaultto/etc/arangodbor/usr/local/etc/arangodb.Thisfileisinstalledwhenusingapackagemanagerlikerpmordpkg.IfyoumodifythisfileandlaterupgradetoanewversionofArangoDB,thenthepackagemanagernormallywarnsyouabouttheconflict.Inordertoavoidthesewarningforsmalladjustments,youcanputlocaloverridesintoafilearangod.conf.local.
Onlycommandlineoptionswithavalueshouldbesetwithintheconfigurationfile.Commandlineoptionswhichactasflagsshouldbeenteredonthecommandlinewhenstartingtheserver.
Eachoptionisspecifiedonaseparatelineintheform:
key=value
Alternatively,aheadersectioncanbespecifiedandoptionspertainingtothatsectioncanbespecifiedinashorterform
[log]
level=trace
ratherthanspecifying
log.level=trace
Soyouseeingeneral--section.paramvaluetranslatesto
[section]
param=value
ServerConfiguration
381
Whitespacearound=isignoredintheconfigurationfile.Donotputspacesaroundadditional=intheparametervaluehowever.Thefollowingexampleshowsthecorrectwaytospecifyalogleveloftraceforthetopicstartup:jslog.level=startup=traceNotethatthereisnowhitespacebetweenstartupand=,andalsonot=andtrace.
Whereonesectionmayoccurmultipletimes,andthelastoccuranceofparamwillbecomethefinalvalue.Incaseofparametersbeingvectors,multipleoccuranceaddsanotheritemtothevector.Vectorscanbeidentifiedbythe...inthe--helpoutputofthebinaries.
Commentscanbeplacedintheconfigurationfile,onlyifthelinebeginswithoneormorehashsymbols(#).
Theremaybeoccasionswhereaconfigurationfileexistsandtheuserwishestooverrideconfigurationsettingsstoredinaconfigurationfile.Anysettingsspecifiedonthecommandlinewilloverwritethesamesettingwhenitappearsinaconfigurationfile.Iftheuserwishestocompletelyignoreconfigurationfileswithoutnecessarilydeletingthefile(orfiles),thenaddthecommandlineoption
-cnone
or
--configurationnone
whenstartinguptheserver.Notethat,thewordnoneiscase-insensitive.
ServerConfiguration
382
OperatingSystemConfiguration
FileSystems
(LINUX)
WerecommendtonotuseBTRFSonlinux,it'sknowntonotworkwellinconjunctionwithArangoDB.WeexperiencedthatarangodbfacinglatencyissuesonaccessingitsdatabasefilesonBTRFSpartitions.InconjunctionwithBTRFSandAUFSwealsosawdatalossonrestart.
VirtualMemoryPageSizes(LINUX)
Bydefault,ArangoDBusesJemallocasthememoryallocator.Jemallocdoesagoodjobofreducingvirtualmemoryfragmentation,especiallyforlong-runningprocesses.Unfortunately,someOSconfigurationscaninterferewithJemalloc'sabilitytofunctionproperly.Specifically,Linux's"transparenthugepages",Windows'"largepages"andothersimilarfeaturessometimespreventJemallocfromreturningunusedmemorytotheoperatingsystemandresultinunnecessarilyhighmemoryuse.Therefore,werecommenddisablingthesefeatureswhenusingJemallocwithArangoDB.Pleaseconsultyouroperatingsystem'sdocumentationforhowtodothis.
Execute
sudobash-c"echomadvise>/sys/kernel/mm/transparent_hugepage/enabled"
sudobash-c"echomadvise>/sys/kernel/mm/transparent_hugepage/defrag"
beforeexecutingarangod.
SwapSpace
(LINUX)
Itisrecommendedtoassignswapspaceforaserverthatisrunningarangod.Configuringswapspacecanpreventtheoperatingsystem'sOOMkillerfromkillingArangoDBtooeagerlyonLinux.
Over-CommitMemory
FortheMMFilesstorageengine,execute
sudobash-c"echo0>/proc/sys/vm/overcommit_memory"
beforeexecutingarangod.
FortheRocksDBstorageengine,execute
sudobash-c"echo2>/proc/sys/vm/overcommit_memory"
beforestarting.
Fromwww.kernel.org:
Whenthisflagis0,thekernelattemptstoestimatetheamountoffreememoryleftwhenuserspacerequestsmorememory.
Whenthisflagis1,thekernelpretendsthereisalwaysenoughmemoryuntilitactuallyrunsout.
Whenthisflagis2,thekernelusesa"neverovercommit"policythatattemptstopreventanyovercommitofmemory.
OperatingSystemConfiguration
383
Notethatthenusinganovercommit_memorysettingof2,thiswillbydefaultallowprocessestouseallswapspacebutonlyhalfoftheavailableRAM.Thiscanbechangedbyadjustingthevalueofovercommit_ratioaswell.
Fromwww.kernel.org:
Whenovercommit_memoryissetto2,thecommittedaddressspaceisnotpermittedtoexceedswapplusthispercentageofphysicalRAM.
ZoneReclaim
Execute
sudobash-c"echo0>/proc/sys/vm/zone_reclaim_mode"
beforeexecutingarangod.
Fromwww.kernel.org:
ThisisvalueORedtogetherof
1=Zonereclaimon2=Zonereclaimwritesdirtypagesout4=Zonereclaimswapspages
NUMAMulti-processorsystemsoftenhavenon-uniformAccessMemory(NUMA).ArangoDBshouldbestartedwithinterleaveonsuchsystem.Thiscanbeachievedusing
numactl--interleave=allarangod...
MaxMemoryMappings(LINUX)
Linuxkernelsbydefaultrestrictthemaximumnumberofmemorymappingsofasingleprocesstoabout64Kmappings.Whilethisvalueissufficientformostworkloads,itmaybetoolowforaprocessthathaslotsofparallelthreadsthatallrequiretheirownmemorymappings.Inthiscaseallthethreads'memorymappingswillbeaccountedtothesinglearangodprocess,andthemaximumnumberof64Kmappingsmaybereached.Whenthemaximumnumberofmappingsisreached,callstommapwillfail,sotheprocesswillthinknomorememoryisavailablealthoughtheremaybeplentyofRAMleft.
Toavoidthisscenario,itisrecommendedtoraisethedefaultvalueforthemaximumnumberofmemorymappingstoasufficientlyhighvalue.Asaruleofthumb,onecoulduse8timesthenumberofavailablecorestimes8,000.
Fora32coreserver,agoodrule-of-thumbvaluethuswouldbe2,048,000(3288000).Tosetthevalueonce,usethefollowingcommandbeforestartingarangod:
sudobash-c"sysctl-w'vm.max_map_count=2048000'"
Tomakethesettingsdurable,itwillbenecessarytostoretheadjustedsettingsin/etc/sysctl.conforotherplacesthattheoperatingsystemislookingat.
EnvironmentVariables(LINUX)
ItisrecommendedtosettheenvironmentvariableGLIBCXX_FORCE_NEWto1onsystemsthatuseglibc++inordertodisablethememorypoolingbuiltintoglibc++.ThatmemorypoolingisunnecessarybecauseJemallocwillalreadydomemorypooling.
OperatingSystemConfiguration
384
Execute
exportGLIBCXX_FORCE_NEW=1
beforestartingarangod.
32bitWhileitispossibletocompileArangoDBon32bitsystem,thisisnotarecommendedenvironment.64bitsystemscanaddressasignificantlybiggermemoryregion.
OperatingSystemConfiguration
385
ManagingEndpointsTheArangoDBservercanlistenforincomingrequestsonmultipleendpoints.
TheendpointsarenormallyspecifiedeitherinArangoDB'sconfigurationfileoronthecommand-line,usingthe--server.endpoint.ArangoDBsupportsdifferenttypesofendpoints:
tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryptionssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryptionunix:///path/to/socket-Unixdomainsocketendpoint
IfaTCP/IPendpointisspecifiedwithoutaportnumber,thenthedefaultport(8529)willbeused.Ifmultipleendpointsneedtobeused,theoptioncanberepeatedmultipletimes.
ThedefaultendpointforArangoDBistcp://127.0.0.1:8529ortcp://localhost:8529.
EXAMPLES
unix>./arangod--server.endpointtcp://127.0.0.1:8529
--server.endpointssl://127.0.0.1:8530
--ssl.keyfileserver.pem/tmp/vocbase
2012-07-26T07:07:47Z[8161]INFOusingSSLprotocolversion'TLSv1'
2012-07-26T07:07:48Z[8161]INFOusingendpoint'ssl://127.0.0.1:8530'forhttpsslrequests
2012-07-26T07:07:48Z[8161]INFOusingendpoint'tcp://127.0.0.1:8529'forhttptcprequests
2012-07-26T07:07:49Z[8161]INFOArangoDB(version1.1.alpha)isreadyforbusiness
2012-07-26T07:07:49Z[8161]INFOHaveFun!
TCPEndpoints
Givenahostname:
--server.endpointtcp://hostname:port
GivenanIPv4address:
--server.endpointtcp://ipv4-address:port
GivenanIPv6address:
--server.endpointtcp://[ipv6-address]:port
Ononespecificethernetinterfaceeachportcanonlybeboundexactlyonce.YoucanlookupyouravailableinterfacesusingtheifconfigcommandonLinux/MacOSX-theWindowsequivalentisipconfig(SeeWikipediaformoredetails).ThegeneralnamesoftheinterfacesdifferonOS'sandhardwarestheyrunon.However,typicallyeveryhosthasasocalledloopbackinterface,whichisavirtualinterface.Byconventionitalwayshastheaddress127.0.0.1or::1(ipv6),andcanonlybereachedfromexactlytheverysamehost.Ethernetinterfacesusuallyhavenameslikeeth0,wlan0,eth1:17,le0oraplaintextnameinWindows.
Tofindoutwhichservicesalreadyuseports(soArangoDBcan'tbindthemanymore),youcanusethenetstatcommand(itbehavesalittledifferentoneachplatform,runitwith-lnptonLinux,-ptcponMacOSXorwith-anonwindowsforvaluableinformation).
ArangoDBcanalsodoasocalledbroadcastbindusingtcp://0.0.0.0:8529.Thiswayitwillbereachableonallinterfacesofthehost.Thismaybeusefulondevelopmentsystemsthatfrequentlychangetheirnetworksetuplikelaptops.
SpecialnoteonIPv6link-localaddresses
ArangoDBcanalsolistentoIPv6link-localaddressesviaaddingthezoneIDtotheIPv6addressintheform[ipv6-link-local-address%zone-id].However,whatyouprobablyinsteadwantistobindtoalocalIPv6address.LocalIPv6addressesstartwithfd.Ifyouonlyseeafe80:IPv6addressinyourinterfaceconfigurationbutnoIPv6addressstartingwithfdyourinterfacehasnolocalIPv6addressassigned.YoucanreadmoreaboutIPv6link-localaddresseshere.
Example
ManagingEndpoints
386
Bindtoalink-localandlocalIPv6address.
unix>ifconfig
Thiscommandlistsallinterfacesandassignedipaddresses.Thelink-localaddressmaybefe80::6257:18ff:fe82:3ec6%eth0(IPv6addressplusinterfacename).AlocalIPv6addressmaybefd12:3456::789a.TobindArangoDBtoitstartarangodwith--server.endpointtcp://[fe80::6257:18ff:fe82:3ec6%eth0]:8529.Usetelnettotesttheconnection.
unix>telnetfe80::6257:18ff:fe82:3ec6%eth08529
Tryingfe80::6257:18ff:fe82:3ec6...
Connectedtomy-machine.
Escapecharacteris'^]'.
GET/HTTP/1.1
HTTP/1.1301MovedPermanently
Location:/_db/_system/_admin/aardvark/index.html
Content-Type:text/html
Server:ArangoDB
Connection:Keep-Alive
Content-Length:197
<html><head><title>Moved</title></head><body><h1>Moved</h1><p>Thispagehasmovedto<ahref="/_db/_system/_admin/aardvark/inde
x.html">/_db/_system/_admin/aardvark/index.html</a>.</p></body></html>
Reuseaddress--tcp.reuse-address
IfthisbooleanoptionissettotruethenthesocketoptionSO_REUSEADDRissetonallserverendpoints,whichisthedefault.Ifthisoptionissettofalseitispossiblethatittakesuptoaminuteafteraserverhasterminateduntilitispossibleforanewservertousethesameendpointagain.Thisiswhythisisactivatedbydefault.
Pleasenotehoweverthatundersomeoperatingsystemsthiscanbeasecurityriskbecauseitmightbepossibleforanotherprocesstobindtothesameaddressandport,possiblyhijackingnetworktraffic.UnderWindows,ArangoDBadditionallysetstheflagSO_EXCLUSIVEADDRUSEasameasuretoalleviatethisproblem.
Backlogsize--tcp.backlog-size
AllowstospecifythesizeofthebacklogforthelistensystemcallThedefaultvalueis10.Themaximumvalueisplatform-dependent.Specifyingahighervaluethandefinedinthesystemheader'sSOMAXCONNmayresultinawarningonserverstart.Theactualvalueusedbylistenmayalsobesilentlytruncatedonsomeplatforms(thishappensinsidethelistensystemcall).
ManagingEndpoints
387
SSLConfiguration
SSLEndpoints
Givenahostname:
--server.endpointtcp://hostname:port
GivenanIPv4address:
--server.endpointtcp://ipv4-address:port
GivenanIPv6address:
--server.endpointtcp://[ipv6-address]:port
Note:IfyouareusingSSL-encryptedendpoints,youmustalsosupplythepathtoaservercertificateusingthe--ssl.keyfileoption.
Keyfile--ssl.keyfilefilename
IfSSLencryptionisused,thisoptionmustbeusedtospecifythefilenameoftheserverprivatekey.ThefilemustbePEMformattedandcontainboththecertificateandtheserver'sprivatekey.
Thefilespecifiedbyfilenamecanbegeneratedusingopenssl:
#createprivatekeyinfile"server.key"
opensslgenrsa-des3-outserver.key1024
#createcertificatesigningrequest(csr)infile"server.csr"
opensslreq-new-keyserver.key-outserver.csr
#copyawayoriginalprivatekeyto"server.key.org"
cpserver.keyserver.key.org
#removepassphrasefromtheprivatekey
opensslrsa-inserver.key.org-outserver.key
#signthecsrwiththekey,createscertificatePEMfile"server.crt"
opensslx509-req-days365-inserver.csr-signkeyserver.key-outserver.crt
#combinecertificateandkeyintosinglePEMfile"server.pem"
catserver.crtserver.key>server.pem
YoumayusecertificatesissuedbyaCertificateAuthorityorself-signedcertificates.Self-signedcertificatescanbecreatedbyatoolofyourchoice.WhenusingOpenSSLforcreatingtheself-signedcertificate,thefollowingcommandsshouldcreateavalidkeyfile:
-----BEGINCERTIFICATE-----
(base64encodedcertificate)
-----ENDCERTIFICATE-----
-----BEGINRSAPRIVATEKEY-----
(base64encodedprivatekey)
-----ENDRSAPRIVATEKEY-----
Forfurtherinformationpleasecheckthemanualsofthetoolsyouusetocreatethecertificate.
CAFile--ssl.cafilefilename
SSLConfiguration
388
ThisoptioncanbeusedtospecifyafilewithCAcertificatesthataresenttotheclientwhenevertheserverrequestsaclientcertificate.Ifthefileisspecified,TheserverwillonlyacceptclientrequestswithcertificatesissuedbytheseCAs.Donotspecifythisoptionifyouwantclientstobeabletoconnectwithoutspecificcertificates.
ThecertificatesinfilenamemustbePEMformatted.
SSLprotocol
--ssl.protocolvalue
Usethisoptiontospecifythedefaultencryptionprotocoltobeused.Thefollowingvariantsareavailable:
1:SSLv22:SSLv2orSSLv3(negotiated)3:SSLv34:TLSv15:TLSv1.2
Thedefaultvalueis5(TLSv1.2).
SSLcache--ssl.session-cachevalue
SettotrueifSSLsessioncachingshouldbeused.
valuehasadefaultvalueoffalse(i.e.nocaching).
SSLpeercertificate
ThisfeatureisavailableintheEnterpriseEdition.
--ssl.require-peer-certificate
Requireapeercertificatefromtheclientbeforeconnecting.
SSLoptions
--ssl.optionsvalue
ThisoptioncanbeusedtosetvariousSSL-relatedoptions.IndividualoptionvaluesmustbecombinedusingbitwiseOR.
WhichoptionsareavailableonyourplatformisdeterminedbytheOpenSSLversionyouuse.Thelistofoptionsavailableonyourplatformmightberetrievedbythefollowingshellcommand:
>grep"#defineSSL_OP_.*"/usr/include/openssl/ssl.h
#defineSSL_OP_MICROSOFT_SESS_ID_BUG0x00000001L
#defineSSL_OP_NETSCAPE_CHALLENGE_BUG0x00000002L
#defineSSL_OP_LEGACY_SERVER_CONNECT0x00000004L
#defineSSL_OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG0x00000008L
#defineSSL_OP_SSLREF2_REUSE_CERT_TYPE_BUG0x00000010L
#defineSSL_OP_MICROSOFT_BIG_SSLV3_BUFFER0x00000020L
...
AdescriptionoftheoptionscanbefoundonlineintheOpenSSLdocumentation
SSLcipher--ssl.cipher-listcipher-list
ThisoptioncanbeusedtorestricttheservertocertainSSLciphersonly,andtodefinetherelativeusagepreferenceofSSLciphers.
Theformatofcipher-listisdocumentedintheOpenSSLdocumentation.
Tocheckwhichciphersareavailableonyourplatform,youmayusethefollowingshellcommand:
SSLConfiguration
389
>opensslciphers-v
ECDHE-RSA-AES256-SHASSLv3Kx=ECDHAu=RSAEnc=AES(256)Mac=SHA1
ECDHE-ECDSA-AES256-SHASSLv3Kx=ECDHAu=ECDSAEnc=AES(256)Mac=SHA1
DHE-RSA-AES256-SHASSLv3Kx=DHAu=RSAEnc=AES(256)Mac=SHA1
DHE-DSS-AES256-SHASSLv3Kx=DHAu=DSSEnc=AES(256)Mac=SHA1
DHE-RSA-CAMELLIA256-SHASSLv3Kx=DHAu=RSAEnc=Camellia(256)
Mac=SHA1
...
Thedefaultvalueforcipher-listis"ALL".
SSLConfiguration
390
LDAPThisfeatureisavailableintheEnterpriseEdition.
BasicsConcepts
Therearetwomodesofoperation:simpleauthandbind+search.
ThebasicoptionsforspecifyinghowtoaccesstheLDAPserverare--ldap.enabled,--ldap.tls,--ldap.port,--ldap.server.--ldap.serverand--ldap.portcanbereplacedby--ldap.url.Thedefaultfor--ldap.portis389.
Simpleauth
ArangoDBconnectstotheldapserverandauthenticateswiththeusernameandpasswordprovidedbytheAPIauthenticationrequest.IftheLDAPserversuccessfullyverifiesthepasswordthentheuserisauthenticated.
If--ldap.prefixand/or--ldap.suffixisprovided,thenthesimplemodeisselected.
Inordertoauthorizetheuserforoneormoredatabasestherearetwomodesofoperation:databaseattributeorroles.
Databaseattribute
Inthismode,anLDAPattributeoftheuserisusedtospecifytheaccesslevelswithinLDAP.Thedatabase/collectionaccesslevelsinArangoDBarenotused.
--ldap.permissions-attribute-namehastheformatdatabase-name=(*|rw|none)[,database-name=(*|rw|none)].
Example:
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.permissions-attribute-namearangodbPermissions\
--ldap.prefixuid=\
--ldap.suffix,dc=company,dc=com
--ldap.prefixand--ldap.suffixbuildthedistinguishedname(DN).ArangoDBtriestoauthenticatewithprefix+ArangoDBusername+suffixagainsttheLDAPserverandsearchesforthedatabasepermissions.
dn:uid=fermi,dc=example,dc=com
arangodbPermissions:foo=none,bar=rw
ThiswillgiveAdministrateaccesstobarandNoAcesstofoo.Notethatthismethodsonlyallowstospecifydatabaseaccesslevels,notcollectionaccesslevels.
Roles
Inthismode,anLDAPattributeoftheuserisusedtospecifyoneormorerolesforthatusers.Thedatabase/collectionaccesslevelsfortheserolesdefinedinArangoDBarethenused.
Example:
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.roles-attribute-namegroupMembership\
--ldap.prefixuid=\
--ldap.suffix,dc=company,dc=com
--ldap.prefixand--ldap.suffixbuildthedistinguishedname(DN).ArangoDBtrystoauthenticatewithprefix+ArangoDBusername+suffixagainsttheldapserverandsearchesfortherolesintheattributegroupMembership.
dn:uid=fermi,dc=example,dc=com
LDAPOptions
391
groupMembership:project-a
groupMembership:project-b
Thiswillgivethecombinedpermissionsoftherolesproject-aandproject-btotheuser.
Rolestransformationsandfilters
--ldap.roles-includecanbeusedtospecifyaregularexpressionthatisusedtofilterroles.Onlyrolesthatmatchtheregularexpressionareused.
--ldap.roles-excludecanbeusedtospecifyaregularexpressionthatisusedtofilterroles.Onlyrolesthatdonotmatchtheregularexpressionareused.
--ldap.roles-transformationcanbeusedtosepcifyaregularexpressionandreplacementtextas/re/text/.Thisregularexpressionisappliedtotherolenamefound.
--ldap.superuser-rolecanbeusedtospecifytheroleassociatedwiththesuperuser.Anyuserbelongingtothisrolegainssuperuserstatus.Thisroleischeckedbeforeapplyinganyregularexpression.
Example:
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.roles-attribute-namegroupMembership\
--ldap.prefixuid=\
--ldap.suffix,dc=company,dc=com
--ldap.roles-include"^arangodb"
willonlyconsiderrolesthatstartwitharangodb.
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.roles-attribute-namegroupMembership\
--ldap.prefixuid=\
--ldap.suffix,dc=company,dc=com
--ldap.roles-exclude"disabled"
willonlyconsiderrolesthatdocontaintheworddisabled.
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.roles-attribute-namegroupMembership\
--ldap.prefixuid=\
--ldap.suffix,dc=company,dc=com
--ldap.superuser-role"arangodb-admin"
anyonebelongingtothegroup"arangodb-admin"willbecomeasuperuser.
bind+search
Examplewithanonymousauth:
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.basedndc=company,dc=com\
--ldap.permissions-attribute-namearangodbPermissions
WiththisconfigurationArangoDBbindsanonymouslytotheLDAPserverandsearchesfortheuser.IftheuserisfoundaauthenticationisdonewiththeusersDNandpasswordandthendatabasepermissionsarefetched.
ExamplewithDNandpassword:
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.basedndc=company,dc=com\
--ldap.binddncn=admin,dc=company,dc=com\
--ldap.bindpasswdadmin\
--ldap.permissions-attribute-namearangodbPermissions
LDAPOptions
392
WiththisconfigurationArangoDBbindswith--ldap.bindnand--ldap.bindpasswdtotheLDAPserverandsearchesfortheuser.IftheuserisfoundaauthenticationisdonewiththeusersDNandpasswordandthendatabasepermissionsarefetched.
Rolessearch
--ldap.roles-searchsearch-expression
Insteadofspecifyingarolesattributeitispossibletouseasearchwhenusingbind+search.Inthiscasethesearch-expressionmustbeanldapsearchstring.Any{USER}isreplacedbythednoftheuser.
Example:
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.basedndc=company,dc=com\
--ldap.binddncn=admin,dc=company,dc=com\
--ldap.bindpasswdadmin\
--ldap.roles-search'(&(objectClass=groupOfUniqueNames)(uniqueMember={USER}))'
Additionaloptions
--ldap.search-filter"objectClass=*"
Restrictthesearchtospecificobjectclasses.ThedefaultisobjectClass=*.
--ldap.search-attribute"uid"
--ldap.search-attributespecifieswhichattributetocomparewiththeusername.Thedefaultisuid.
--ldap.search-scopesub
`--ldap.search-scopespecifiesinwhichscopetosearchforauser.Validareoneofbase,oneorsub.Thedefaultissub.
ldapurl
--ldap.urlldap://ldap.server.com:1234/dc=example,dc=com?uid?sub
Theldapurlconsistsoftheldapserverandport,abasedn,asearchattributeandascopewhichcanbeoneofbase,oneorsub.
TLSoptions
Aencryptedconnectioncanbeestablishedwith--ldap.tlstrueunderUNIXandGNU/Linuxplatforms.
AllfollowingoptionsarenotavailableunderWindows.
--ldap.tls
Thedefaultisfalse.Withtrueatlsconnectionisestablished.
--ldap.tls-version
Thedefaultis1.2.Availableversionsare1.0,1.1and1.2.
--ldap.tls-cert-check-strategy
Thedefaultishard.Availablestrategiesarenever,hard,demand,allowandtry.
--ldap.tls-cacert-file
LDAPOptions
393
Afilepathtooneormore(concatenated)certificateauthoritycertificatesinpemformat.Asdefaultnofilepathisconfigured.
Followingoptionhasnoeffect/doesnotworkundermacOS.
--ldap.tls-cacert-dir
Adirectorypathtocertificateauthoritycertificatesinc_rehashformat.Asdefaultnodirectorypathisconfigured.
LDAPOptions
394
Command-LineOptionsforLogging
Loglevelsandtopics
ArangoDB'slogoutputisgroupedintotopics.--log.levelcanbespecifiedmultipletimesatstartup,forasmanytopicsasneeded.Thelogverbosityandoutputfilescanbeadjustedperlogtopic.Forexample
--log.levelstartup=trace--log.levelqueries=trace--log.levelinfo
willlogmessagesconcerningstartupattracelevel,AQLqueriesattracelevelandeverythingelseatinfolevel.
Inaconfigurationfile,itiswrittenlikethis:
[log]
level=startup=trace
level=queries=trace
level=info
Notethattheremustnotbeanywhitespacearoundthesecond=.
Theavailableloglevelsare:
fatal:onlylogsfatalerrorserror:onlylogserrorswarning:onlylogswarningsanderrorsinfo:logsinformationmessages,warningsanderrorsdebug:logsdebugandinformationmessages,warningsanderrorstrace:logstrace,debugandinformationmessages,warningsanderrors
Notethatlevelsdebugandtracewillbeveryverbose.
SomerelevantlogtopicsavailableinArangoDB3are:
agency:informationabouttheagencycollector:informationabouttheWALcollector'sstatecompactor:informationaboutthecollectiondatafilecompactordatafiles:datafile-relatedoperationsmmap:informationaboutmemory-mappingoperations(includingmsync)performance:performance-releatedmessagesqueries:executedAQLqueries,slowqueriesreplication:replication-relatedinforequests:HTTPrequestsstartup:informationaboutserverstartupandshutdownthreads:informationaboutthreads
Logoutputs
Thelogoption--log.output<definition>allowsdirectingtheglobalorper-topiclogoutputtodifferentoutputs.Theoutputdefinition<definition>canbeoneof
-forstdin+forstderrsyslog://<syslog-facility>
syslog://<syslog-facility>/<application-name>
file://<relative-path>
Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputfordifferentlogtopics.Tosetupaper-topicoutputconfiguration,use--log.output<topic>=<definition>,e.g.
LoggingOptions
395
queries=file://queries.txt
logsallqueriestothefile"queries.txt".
Theoldoption--log.fileisstillavailablein3.0forconveniencereasons.In3.0itisashortcutforthemoregeneraloption--log.outputfile://filename.
Theoldoption--log.requests-fileisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption--log.outputrequests=file://....
Using--log.outputalsoallowsdirectinglogoutputtodifferentfilesbasedontopics.Forexample,tologallAQLqueriestoafile"queries.log"onecanusetheoptions:
--log.levelqueries=trace--log.outputqueries=file:///path/to/queries.log
ToadditionallylogHTTPrequesttoafilenamed"requests.log"addtheoptions:
--log.levelrequests=info--log.outputrequests=file:///path/to/requests.log
Forcingdirectoutput
Theoption--log.force-directcanbeusedtodisablelogginginanextraloggingthread.Ifsettotrue,anylogmessagesareimmediatelyprintedinthethreadthattriggeredthelogmessage.Thisisnon-optimalforperformancebutcanaiddebugging.Ifsettofalse,logmessagesarehandedofftoanextraloggingthread,whichasynchronouslywritesthelogmessages.
Localtime
Logdatesandtimesinlocaltimezone:--log.use-local-time
Ifspecified,alldatesandtimesinlogmessageswillusetheserver'slocaltime-zone.Ifnotspecified,alldatesandtimesinlogmessageswillbeprintedinUTC/Zulutime.ThedateandtimeformatusedinlogsisalwaysYYYY-MM-DDHH:MM:SS,regardlessofthissetting.IfUTCtimeisused,aZwillbeappendedtoindicateZulutime.
Colorlogging--log.colorvalue
Loggingtoterminaloutputisbydefaultcolored.Colorfulloggingcanbeturnedoffbysettingthevaluetofalse.
SourcefileandLinenumber
Loglinenumber:--log.line-number
Normally,ifanhumanreadablefatal,error,warningorinfomessageislogged,noinformationaboutthefileandlinenumberisprovided.Thefileandlinenumberisonlyloggedfordebugandtracemessage.Thisoptioncanbeusetoalwayslogthesepiecesofinformation.
Prefix
Logprefix:--log.prefixprefix
Thisoptionisusedspecifyanprefixtologgedtext.
Threads
Logthreadidentifier:--log.threadtrue
Wheneverlogoutputisgenerated,theprocessIDiswrittenaspartoftheloginformation.Settingthisoptionappendsthethreadidofthecallingthreadtotheprocessid.Forexample,
2010-09-20T13:04:01Z[19355]INFOreadyforbusiness
LoggingOptions
396
whennothreadisloggedand
2010-09-20T13:04:17Z[19371-18446744072487317056]readyforbusiness
whenthiscommandlineoptionisset.
Toalsologthreadnames,itispossibletosetthe--log.thread-nameoption.Bydefault--log.thread-nameissettofalse.
Role
Logrole:--log.roletrue
Whensettotrue,thisoptionwillmaketheArangoDBloggerprintasinglecharacterwiththeserver'sroleintoeachloggedmessage.Therolesare:
U:undefined/unclear(usedatstartup)S:singleserverC:coordinatorP:primaryA:agent
Thedefaultvalueforthisoptionisfalse,sonoroleswillbelogged.
LoggingOptions
397
GeneralOptions
DatabaseUpgrade
--database.auto-upgrade
Specifyingthisoptionwillmaketheserverperformadatabaseupgradeatstart.AdatabaseupgradewillfirstcomparetheversionnumberstoredinthefileVERSIONinthedatabasedirectorywiththecurrentserverversion.
Ifthetwoversionnumbersmatch,theserverwillstartnormally.
Iftheversionnumberfoundinthedatabasedirectoryishigherthantheversionnumbertheserverisrunning,theserverexpectsthisisanunintentionaldowngradeandwillwarnaboutthis.Itwillhoweverstartnormally.Usingtheserverintheseconditionsishowevernotrecommendednorsupported.
Iftheversionnumberfoundinthedatabasedirectoryislowerthantheversionnumbertheserverisrunning,theserverwillcheckwhetherthereareanyupgradetaskstoperform.Itwillthenexecuteallrequiredupgradetasksandprinttheirstatuses.Ifoneoftheupgradetasksfails,theserverwillexitandrefusetostart.Re-startingtheserverwiththeupgradeoptionwillthenagaintriggertheupgradecheckandexecutionuntiltheproblemisfixed.Ifalltasksarefinished,theserverwillstartnormally.
Whetherornotthisoptionisspecified,theserverwillalwaysperformaversioncheckonstartup.Runningtheserverwithanon-matchingversionnumberintheVERSIONfilewillmaketheserverrefusetostart.
StorageEngine
AsofArangoDB3.2twostorageenginesaresupported.The"traditional"engineiscalledMMFiles,whichisalsothedefaultstorageengine.
AnalternativeenginebasedonRocksDBisalsoprovidedandcanbeturnedonmanually.
Onestorageenginetypeissupportedperserverperinstallation.Liveswitchingofstorageenginesonalreadyinstalledsystemsisn'tsupported.Configuringthewrongengine(notmatchingthepreviouslyusedone)willresultintheserverrefusingtostart.YoumayhoweveruseautotoletArangoDBchoosethepreviouslyusedone.
--server.storage-engine[auto|mmfiles|rocksdb]
Daemon
--daemon
Runstheserverasadaemon(asabackgroundprocess).Thisparametercanonlybesetifthepid(processid)fileisspecified.Thatis,unlessavaluetotheparameterpid-fileisgiven,thentheserverwillreportanerrorandexit.
DefaultLanguage
--default-languagedefault-language
Thedefaultlanguageistusedforsortingandcomparingstrings.Thelanguagevalueisatwo-letterlanguagecode(ISO-639)oritiscomposedbyatwo-letterlanguagecodewithandatwolettercountrycode(ISO-3166).Validlanguagesare"de","en","en_US"or"en_UK".
Thedefaultdefault-languageissettobethesystemlocaleonthatplatform.
Supervisor--supervisor
Executestheserverinsupervisormode.Intheeventthattheserverunexpectedlyterminatesduetoaninternalerror,thesupervisorwillautomaticallyrestarttheserver.Settingthisflagautomaticallyimpliesthattheserverwillrunasadaemon.Notethat,aswiththedaemonflag,thisflagrequiresthatthepid-fileparameterwillset.
unix>./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
2012-06-27T15:58:28Z[10133]INFOstartingupinsupervisormode
GeneralOptions
398
Ascanbeseen(e.g.byexecutingthepscommand),thiswillstartasupervisorprocessandtheactualdatabaseprocess:
unix>psfax|greparangod
10137?Ssl0:00./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
10142?Sl0:00\_./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
Whenthedatabaseprocessterminatesunexpectedly,thesupervisorprocesswillstartupanewdatabaseprocess:
>kill-SIGSEGV10142
>psfax|greparangod
10137?Ssl0:00./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
10168?Sl0:00\_./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
Useridentity
--uiduid
Thename(identity)oftheusertheserverwillrunas.Ifthisparameterisnotspecified,theserverwillnotattempttochangeitsUID,sothattheUIDusedbytheserverwillbethesameastheUIDoftheuserwhostartedtheserver.Ifthisparameterisspecified,thentheserverwillchangeitsUIDafteropeningportsandreadingconfigurationfiles,butbeforeacceptingconnectionsoropeningotherfiles(suchasrecoveryfiles).Thisisusefulwhentheservermustbestartedwithraisedprivileges(incertainenvironments)butsecurityconsiderationsrequirethattheseprivilegesbedroppedoncetheserverhasstartedwork.
Observethatthisparametercannotbeusedtobypassoperatingsystemsecurity.Ingeneral,thisparameter(anditscorrespondingrelativegid)canlowerprivilegesbutnotraisethem.
Groupidentity
--gidgid
Thename(identity)ofthegrouptheserverwillrunas.Ifthisparameterisnotspecified,thentheserverwillnotattempttochangeitsGID,sothattheGIDtheserverrunsaswillbetheprimarygroupoftheuserwhostartedtheserver.Ifthisparameterisspecified,thentheserverwillchangeitsGIDafteropeningportsandreadingconfigurationfiles,butbeforeacceptingconnectionsoropeningotherfiles(suchasrecoveryfiles).
Thisparameterisrelatedtotheparameteruid.
Processidentity
--pid-filefilename
ThenameoftheprocessIDfiletousewhenrunningtheserverasadaemon.Thisparametermustbespecifiedifeithertheflagdaemonorsupervisorisset.
Checkmaxmemorymappings
--server.check-max-memory-mappingscanbeusedonLinuxtomakearangodcheckthenumberofmemorymappingscurrentlyusedbytheprocess(asreportedin/proc//maps)andcompareitwiththemaximumnumberofallowedmappingsasdeterminedby/proc/sys/vm/max_map_count.Ifthecurrentnumberofmemorymappingsgetsnearthemaximumallowedvalue,arangodwilllogawarninganddisallowthecreationoffurtherV8contextstemporarilyuntilthecurrentnumberofmappingsgoesdownagain.
Iftheoptionissettofalse,nosuchcheckswillbeperformed.Allnon-Linuxoperatingsystemsdonotprovidethisoptionandwillignoreit.
Console
--console
Runstheserverinanexclusiveemergencyconsolemode.Whenstartingtheserverwiththisoption,theserverisstartedwithaninteractiveJavaScriptemergencyconsole,withallnetworkingandHTTPinterfacesoftheserverdisabled.
GeneralOptions
399
Norequestscanbemadetotheserverinthismode,andtheonlywaytoworkwiththeserverinthismodeisbyusingtheemergencyconsole.Notethattheservercannotbestartedinthismodeifitisalreadyrunninginthisoranothermode.
RandomGenerator
--random.generatorarg
Theargumentisaninteger(1,2,3or4)whichsetsthemannerinwhichrandomnumbersaregenerated.Thedefaultmethod(3)istousetheanon-blockingrandom(orpseudorandom)numbergeneratorsuppliedbytheoperatingsystem.
Specifyinganargumentof2,usesablockingrandom(orpseudorandom)numbergenerator.Specifyinganargument1setsapseudorandomnumbergeneratorusinganimplicationoftheMersenneTwisterMT19937algorithm.Algorithm4isacombinationoftheblockingrandomnumbergeneratorandtheMersenneTwister.
Enable/disableauthentication
--server.authenticationSettingthisoptiontofalsewillturnoffauthenticationontheserversidesoallclientscanexecuteanyactionwithoutauthorizationandprivilegechecks.Thedefaultvalueistrue.
JWTSecret--server.jwt-secretsecret
ArangoDBwilluseJWTstoauthenticaterequests.UsingthisoptionletsyouspecifyaJWT.
InsingleserversetupsandwhennotspecifyingthissecretArangoDBwillgenerateasecret.
Inclusterdeploymentswhichhaveauthenticationenabledasecretmustbesetconsistentlyacrossallclustertaskssotheycantalktoeachother.
Enable/disableauthenticationforUNIXdomainsockets
--server.authentication-unix-socketsvalue
SettingvaluetotruewillturnoffauthenticationontheserversideforrequestscominginviaUNIXdomainsockets.Withthisflagenabled,clientslocatedonthesamehostastheArangoDBservercanuseUNIXdomainsocketstoconnecttotheserverwithoutauthentication.Requestscominginbyothermeans(e.g.TCP/IP)arenotaffectedbythisoption.
Thedefaultvalueisfalse.
Note:thisoptionisonlyavailableonplatformsthatsupportUNIXdomainsockets.
Enable/disableauthenticationforsystemAPIrequestsonly
--server.authentication-system-onlybooleanControlswhetherincomingrequestsneedauthenticationonlyiftheyaredirectedtotheArangoDB'sinternalAPIsandfeatures,locatedat/_api/,/_admin/etc.Iftheflagissettotrue,thenHTTPauthenticationisonlyrequiredforrequestsgoingtoURLsstartingwith/_,butnotforotherURLs.Theflagcanthusbeusedtoexposeauser-madeAPIwithoutHTTPauthenticationtotheoutsideworld,buttopreventtheoutsideworldfromusingtheArangoDBAPIandtheadmininterfacewithoutauthentication.NotethatcheckingtheURLisperformedafteranydatabasenameprefixhasbeenremoved.ThatmeanswhentheactualURLcalledis/_db/_system/myapp/myaction,theURL/myapp/myactionwillbeusedforauthentication-system-onlycheck.Thedefaultistrue.NotethatauthenticationstillneedstobeenabledfortheserverregularlyinorderforHTTPauthenticationtobeforcedfortheArangoDBAPIandthewebinterface.Settingonlythisflagisnotenough.YoucancontrolArangoDB'sgeneralauthenticationfeaturewiththe--server.authenticationflag.
Enableauthenticationcachetimeout
--server.authentication-timeoutvalue
Setsthecachetimeouttovalue(inseconds).ThisisonlynecessaryifyouuseanexternalauthenticationsystemlikeLDAP.
Enablelocalauthentication--server.local-authenticationvalue
GeneralOptions
400
Ifsettofalseonlyusetheexternalauthenticationsystem.Iftruealsousethelocal_userscollections.
Thedefaultvalueistrue.
Enable/disablereplicationapplier
--database.replication-applierflag
Iffalsetheserverwillstartwithreplicationappliersturnedoff,evenifthereplicationappliersareconfiguredwiththeautoStartoption.Usingthecommand-lineoptionwillnotchangethevalueoftheautoStartoptionintheapplierconfiguration,butwillsuppressauto-startingthereplicationapplierjustonce.
Iftheoptionisnotused,ArangoDBwillreadtheapplierconfigurationfromthefileREPLICATION-APPLIER-CONFIGonstartup,andusethevalueoftheautoStartattributefromthisfile.
Thedefaultistrue.
Keep-alivetimeout--http.keep-alive-timeout
AllowstospecifythetimeoutforHTTPkeep-aliveconnections.Thetimeoutvaluemustbespecifiedinseconds.Idlekeep-aliveconnectionswillbeclosedbytheserverautomaticallywhenthetimeoutisreached.Akeep-alive-timeoutvalue0willdisablethekeepalivefeatureentirely.
HideProductheader
--http.hide-product-header
Iftrue,theserverwillexcludetheHTTPheader"Server:ArangoDB"inHTTPresponses.Ifsettofalse,theserverwillsendtheheaderinresponses.
Thedefaultisfalse.
Allowmethodoverride--http.allow-method-override
Whenthisoptionissettotrue,theHTTPrequestmethodwilloptionallybefetchedfromoneofthefollowingHTTPrequestheadersifpresentintherequest:
x-http-methodx-http-method-overridex-method-override
Iftheoptionissettotrueandanyoftheseheadersisset,therequestmethodwillbeoverriddenbythevalueoftheheader.Forexample,thisallowsissuinganHTTPDELETErequestwhichtotheoutsideworldwilllooklikeanHTTPGETrequest.Thisallowsbypassingproxiesandtoolsthatwillonlyletcertainrequesttypespass.
Settingthisoptiontotruemayimposeasecurityrisksoitshouldonlybeusedincontrolledenvironments.
Thedefaultvalueforthisoptionisfalse.
Serverthreads
--server.threadsnumber
Specifiesthenumberofthreadsthatarespawnedtohandlerequests.
Togglingserverstatistics
--server.statisticsvalue
Ifthisoptionisvalueisfalse,thenArangoDB'sstatisticsgatheringisturnedoff.StatisticsgatheringcausesregularCPUactivitysousingthisoptiontoturnitoffmightrelieveheavy-loadedinstancesabit.
GeneralOptions
401
Sessiontimeout
timetoliveforserversessions--server.session-timeoutvalue
Thetimeoutforwebinterfacesessions,usingforauthenticatingrequeststothewebinterface(/_admin/aardvark)andrelatedareas.
Sessionsareonlyusedwhenauthenticationisturnedon.
Foxxqueues
enableordisabletheFoxxqueuesfeature--foxx.queuesflagIftrue,theFoxxqueueswillbeavailableandjobsinthequeueswillbeexecutedasynchronously.Thedefaultistrue.Whensettofalsethequeuemanagerwillbedisabledandanyjobsarepreventedfrombeingprocessed,whichmayreduceCPUloadabit.
Foxxqueuespollinterval
pollintervalforFoxxqueues--foxx.queues-poll-intervalvalueThepollintervalfortheFoxxqueuesmanager.Thevalueisspecifiedinseconds.LowervalueswillmeanmoreimmediateandmorefrequentFoxxqueuejobexecution,butwillmakethequeuethreadwakeupandquerythequeuesmoreoften.Whensettoalowvalue,thequeuethreadmightcauseCPUload.Thedefaultis1second.IfFoxxqueuesarenotusedmuch,thenthisvaluemaybeincreasedtomakethequeuesthreadwakeupless.
Directory
--database.directorydirectory
Thedirectorycontainingthecollectionsanddatafiles.Defaultsto/var/lib/arango.Whenspecifyingthedatabasedirectory,pleasemakesurethedirectoryisactuallywritablebythearangodprocess.
YoushouldfurthernotuseadatabasedirectorywhichisprovidedbyanetworkfilesystemsuchasNFS.Thereasonisthatnetworkedfilesystemsmightcauseinconsistencieswhentherearemultipleparallelreadersorwritersortheylackfeaturesrequiredbyarangod(e.g.flock()).
directory
Whenusingthecommandlineversion,youcansimplysupplythedatabasedirectoryasargument.
Examples
>./arangod--server.endpointtcp://127.0.0.1:8529--database.directory
/tmp/vocbase
Journalsize
--database.maximal-journal-sizesizeMaximalsizeofjournalinbytes.Canbeoverwrittenwhencreatinganewcollection.Notethatthisalsolimitsthemaximalsizeofasingledocument.Thedefaultis32MB.
Waitforsync
defaultwaitforsyncbehavior--database.wait-for-syncbooleanDefaultwait-for-syncvalue.Canbeoverwrittenwhencreatinganewcollection.Thedefaultisfalse.
Forcesyncingofproperties
forcesyncingofcollectionpropertiestodisk--database.force-sync-propertiesbooleanForcesyncingofcollectionpropertiestodiskaftercreatingacollectionorupdatingitsproperties.Ifturnedoff,nofsyncwillhappenforthecollectionanddatabasepropertiesstoredinparameter.jsonfilesinthefilesystem.Turningoffthisoptionwillspeedupworkloadsthatcreateanddropalotofcollections(e.g.testsuites).Thedefaultistrue.
LimitingmemoryforAQLqueries
--query.memory-limitvalue
GeneralOptions
402
Thedefaultmaximumamountofmemory(inbytes)thatasingleAQLquerycanuse.WhenasingleAQLqueryreachesthespecifiedlimitvalue,thequerywillbeabortedwitharesourcelimitexceededexception.Inacluster,thememoryaccountingisdonepershard,sothelimitvalueiseffectivelyamemorylimitperquerypershard.
ThegloballimitvaluecanbeoverridenperquerybysettingthememoryLimitoptionvalueforindividualquerieswhenrunninganAQLquery.
Thedefaultvalueis0,meaningthatthereisnomemorylimit.
TurningAQLwarningsintoerrors--query.fail-on-warningvalue
Whensettotrue,AQLqueriesthatproducewarningswillinstantlyabortandthrowanexception.ThisoptioncanbesettocatchobviousissueswithAQLqueriesearly.Whensettofalse,AQLqueriesthatproducewarningswillnotabortandreturnthewarningsalongwiththequeryresults.TheoptioncanalsobeoverriddenforeachindividualAQLquery.
Enable/disableAQLquerytracking
--query.trackingflag
Iftrue,theserver'sAQLslowquerytrackingfeaturewillbeenabledbydefault.Trackingofqueriescanbedisabledbysettingtheoptiontofalse.
Thedefaultistrue.
Enable/disabletrackingofbindvariablesinAQLqueries
--query.tracking-with-bindvarsflag
Iftrue,thenthebindvariableswillbetrackedforallrunningandslowAQLqueries.Thisoptiononlyhasaneffectif--query.trackingwassettotrue.Trackingofbindvariablescanbedisabledbysettingtheoptiontofalse.
Thedefaultistrue.
ThresholdforslowAQLqueries--query.slow-thresholdvalue
BysettingvalueitcanbecontrolledafterwhatexecutiontimeanAQLqueryisconsidered"slow".Anyslowqueriesthatexceedtheexecutiontimespecifiedinvaluewillbeloggedwhentheyarefinished.Thethresholdvalueisspecifiedinseconds.Trackingofslowqueriescanbeturnedoffentirelybysettingtheoption--query.trackingtofalse.
Thedefaultvalueis10.0.
Throwcollectionnotloadederror--database.throw-collection-not-loaded-errorflag
Accessinganot-yetloadedcollectionwillautomaticallyloadacollectiononfirstaccess.Thisflagcontrolswhathappensincaseanoperationwouldneedtowaitforanotherthreadtofinalizeloadingacollection.Ifsettotrue,thenthefirstoperationthataccessesanunloadedcollectionwillloadit.Furtherthreadsthattrytoaccessthesamecollectionwhileitisstillloadingwillgetanerror(1238,collectionnotloaded).Whentheinitialoperationhascompletedloadingthecollection,alloperationsonthecollectioncanbecarriedoutnormally,anderror1238willnotbethrown.
Ifsettofalse,thefirstthreadthataccessesanot-yetloadedcollectionwillstillloadit.Otherthreadsthattrytoaccessthecollectionwhileloadingwillnotfailwitherror1238butinsteadblockuntilthecollectionisfullyloaded.Thisconfigurationmightleadtoallserverthreadsbeingblockedbecausetheyareallwaitingforthesamecollectiontocompleteloading.Settingtheoptiontotruewillpreventthisfromhappening,butrequiresclientstocatcherror1238andreactonit(maybebyschedulingaretryforlater).
Thedefaultvalueisfalse.
AQLQuerycachingmode
GeneralOptions
403
--query.cache-mode
TogglestheAQLquerycachebehavior.Possiblevaluesare:
off:donotusequerycacheon:alwaysusequerycache,exceptforqueriesthathavetheircacheattributesettofalsedemand:usequerycacheonlyforqueriesthathavetheircacheattributesettotrue
AQLQuerycachesize
--query.cache-entries
Maximumnumberofqueryresultsthatcanbestoredperdatabase-specificquerycache.Ifaqueryiseligibleforcachingandthenumberofitemsinthedatabase'squerycacheisequaltothisthresholdvalue,anothercachedqueryresultwillberemovedfromthecache.
Thisoptiononlyhasaneffectifthequerycachemodeissettoeitheronordemand.
JavaScriptcodeexecution--javascript.allow-admin-execute
Thisoptioncanbeusedtocontrolwhetheruser-definedJavaScriptcodeisallowedtobeexecutedonserverbysendingviaHTTPtotheAPIendpoint/_admin/executewithanauthenticateduseraccount.Thedefaultvalueisfalse,whichdisablestheexecutionofuser-definedcode.Thisisalsotherecommendedsettingforproduction.Intestenvironments,itmaybeconvenienttoturntheoptiononinordertosendarbitrarysetuporteardowncommandsforexecutionontheserver.
V8contexts
--javascript.v8-contextsnumber
SpecifiesthemaximumnumberofV8contextsthatarecreatedforexecutingJavaScriptcode.MorecontextsallowexecutingmoreJavaScriptactionsinparallel,providedthattherearealsoenoughthreadsavailable.PleasenotethateachV8contextwilluseasubstantialamountofmemoryandrequiresperiodicCPUprocessingtimeforgarbagecollection.
NotethatthisvalueconfiguresthemaximumnumberofV8contextsthatcanbeusedinparallel.UponserverstartonlyasmanyV8contextswillbecreatedasareconfiguredinoption--javascript.v8-contexts-minimum.TheactualnumberofavailableV8contextsmayfloatatruntimebetween--javascript.v8-contexts-minimumand--javascript.v8-contexts.WhenthereareunusedV8contextsthatlingeraround,theserver'sgarbagecollectorthreadwillautomaticallydeletethem.
--javascript.v8-contexts-minimumnumber
SpecifiestheminimumnumberofV8contextsthatwillbepresentatanytimetheserverisrunning.TheactualnumberofV8contextswillneverdropbelowthisvalue,butitmaygoupashighasspecifiedviatheoption--javascript.v8-contexts.
WhenthereareunusedV8contextsthatlingeraroundandthenumberofV8contextsisgreaterthan--javascript.v8-contexts-minimumtheserver'sgarbagecollectorthreadwillautomaticallydeletethem.
--javascript.v8-contexts-max-invocations
SpecifiesthemaximumnumberofinvocationsafterwhichausedV8contextisdisposed.Thedefaultvalueof--javascript.v8-contexts-max-invocationsis0,meaningthatthemaximumnumberofinvocationspercontextisunlimited.
--javascript.v8-contexts-max-age
Specifiesthetimeduration(inseconds)afterwhichtimeaV8contextisdisposedautomaticallyafteritscreation.Ifthetimeiselapsed,thecontextwillbedisposed.Thedefaultvaluefor--javascript.v8-contexts-max-ageis60seconds.
Ifboth--javascript.v8-contexts-max-invocationsand--javascript.v8-contexts-max-ageareset,thenthecontextwillbedestroyedwheneitherofthespecifiedthresholdvaluesisreached.
Garbagecollectionfrequency(time-based)
--javascript.gc-frequencyfrequency
Specifiesthefrequency(inseconds)fortheautomaticgarbagecollectionofJavaScriptobjects.Thissettingisusefultohavethegarbagecollectionstillworkinperiodswithnoorlittlenumbersofrequests.
GeneralOptions
404
Garbagecollectioninterval(request-based)
--javascript.gc-intervalinterval
Specifiestheinterval(approximatelyinnumberofrequests)thatthegarbagecollectionforJavaScriptobjectswillberunineachthread.
V8options--javascript.v8-optionsoptions
OptionalargumentstopasstotheV8Javascriptengine.TheV8enginewillrunwithdefaultsettingsunlessexplicitoptionsarespecifiedusingthisoption.TheoptionspassedwillbeforwardedtotheV8enginewhichwillparsethemonitsown.Passinginvalidoptionsmayresultinanerrorbeingprintedonstderrandtheoptionbeingignored.
Optionsneedtobepassedinonestring,withV8optionnamesbeingprefixedwithdoubledashes.Multipleoptionsneedtobeseparatedbywhitespace.TogetalistofallavailableV8options,youcanusethevalue"--help" asfollows:
--javascript.v8-options="--help"
AnotherexampleofspecificV8optionsbeingsetatstartup:
--javascript.v8-options="--log"
NamesandfeaturesorusableoptionsdependontheversionofV8beingused,andmightchangeinthefutureifadifferentversionofV8isbeingusedinArangoDB.NotalloptionsofferedbyV8mightbesensibletouseinthecontextofArangoDB.Usethespecificoptionsonlyifyouaresurethattheyarenotharmfulfortheregulardatabaseoperation.
GeneralOptions
405
MMFilesWrite-aheadlogoptionsSinceArangoDB2.2,theMMFilesstorageenginewillwritealldata-modificationoperationsintoitswrite-aheadlog.
WithArangoDB3.2anotherStorageengineoptionbecomesavailable-RocksDB.IncaseofusingRocksDBmostofthesubsequentoptionsdon'thaveausefulmeaning.
Thewrite-aheadlogisasequenceoflogfilesthatarewritteninanappend-onlyfashion.Fulllogfileswilleventuallybegarbage-collected,andtherelevantdatamightbetransferredintocollectionjournalsanddatafiles.Unneededandalreadygarbage-collectedlogfileswilleitherbedeletedorkeptforthepurposeofkeepingareplicationbacklog.
Directory
TheWALlogfilesdirectory:--wal.directory
Specifiesthedirectoryinwhichthewrite-aheadlogfilesshouldbestored.Ifthisoptionisnotspecified,itdefaultstothesubdirectoryjournalsintheserver'sglobaldatabasedirectory.Ifthedirectoryisnotpresent,itwillbecreated.
Logfilesize
thesizeofeachWALlogfile--wal.logfile-sizeSpecifiesthefilesize(inbytes)foreachwrite-aheadlogfile.Thelogfilesizeshouldbechosensothateachlogfilecanstoreaconsiderableamountofdocuments.Thebiggerthelogfilesizeischosen,thelongeritwilltaketofillupasinglelogfile,whichalsoinfluencesthedelayuntilthedatainalogfilewillbegarbage-collectedandwrittentocollectionjournalsanddatafiles.Italsoaffectshowlonglogfilerecoverywilltakeatserverstart.
Allowoversizeentries
whetherornotoversizeentriesareallowed--wal.allow-oversize-entriesWhetherornotitisallowedtostoreindividualdocumentsthatarebiggerthanwouldfitintoasinglelogfile.Settingtheoptiontofalsewillmakesuchoperationsfailwithanerror.Settingtheoptiontotruewillmakesuchoperationssucceed,butwithahighpotentialperformanceimpact.Thereasonisthatforeachoversizeoperation,anindividualoversizelogfileneedstobecreatedwhichmayalsoblockotheroperations.Theoptionshouldbesettofalseifitiscertainthatdocumentswillalwayshaveasizesmallerthanasinglelogfile.
Numberofreservelogfiles
maximumnumberofreservelogfiles--wal.reserve-logfilesThemaximumnumberofreservelogfilesthatArangoDBwillcreateinabackgroundprocess.Reservelogfilesareusefulinthesituationwhenanoperationneedstobewrittentoalogfilebutthereservespaceinthelogfileistoolowforstoringtheoperation.Inthiscase,anewlogfileneedstobecreatedtostoretheoperation.Creatingnewlogfilesisnormallyslow,soArangoDBwilltrytopre-createlogfilesinabackgroundprocesssotherearealwaysreservelogfileswhentheactivelogfilegetsfull.ThenumberofreservelogfilesthatArangoDBkeepsinthebackgroundisconfigurablewiththisoption.
Numberofhistoriclogfiles
maximumnumberofhistoriclogfiles--wal.historic-logfilesThemaximumnumberofhistoriclogfilesthatArangoDBwillkeepaftertheyhavebeengarbage-collected.Ifnoreplicationisused,thereisnoneedtokeephistoriclogfilesexceptforhavingalocalchangelog.Inareplicationsetup,thenumberofhistoriclogfilesaffectstheamountofdataaslavecanfetchfromthemaster'slogs.Themorehistoriclogfiles,themorehistoricdataisavailableforaslave,whichisusefuliftheconnectionbetweenmasterandslaveisunstableorslow.Nothavingenoughhistoriclogfilesavailablemightleadtologfiledatabeingdeletedonthemasteralreadybeforeaslavehasfetchedit.
Syncinterval
intervalforautomatic,non-requesteddisksyncs--wal.sync-intervalTheinterval(inmilliseconds)thatArangoDBwillusetoautomaticallysynchronizedatainitswrite-aheadlogstodisk.Automaticsyncswillonlybeperformedfornot-yetsynchronizeddata,andonlyforoperationsthathavebeenexecutedwithoutthewaitForSyncattribute.
Flushtimeout
Write-AheadLogOptions
406
WALflushtimeout`--wal.flush-timeoutThetimeout(inmilliseconds)thatArangoDBwillatmostwaitwhenflushingafullWALlogfiletodisk.Whenthetimeoutisreachedandtheflushisnotcompleted,theoperationthatrequestedtheflushwillfailwithalocktimeouterror.
Throttling
ThrottlewritestoWALwhenatleastsuchmanyoperationsarewaitingforgarbagecollection:--wal.throttle-when-pending
Themaximumvalueforthenumberofwrite-aheadloggarbage-collectionqueueelements.Ifsetto0,thequeuesizeisunbounded,andnowrite-throttlingwilloccur.Ifsettoanon-zerovalue,write-throttlingwillautomaticallykickinwhenthegarbage-collectionqueuecontainsatleastasmanyelementsasspecifiedbythisoption.Whilewrite-throttlingisactive,data-modificationoperationswillintentionallybedelayedbyaconfigurableamountoftime.Thisistoensurethewrite-aheadloggarbagecollectorcancatchupwiththeoperationsexecuted.Write-throttlingwillstayactiveuntilthegarbage-collectionqueuesizegoesdownbelowthespecifiedvalue.Write-throttlingisturnedoffbydefault.
--wal.throttle-wait
Thisoptiondeterminesthemaximumwaittime(inmilliseconds)foroperationsthatarewrite-throttled.Ifwrite-throttlingisactiveandanewwriteoperationistobeexecuted,itwillwaitforatmostthespecifiedamountoftimeforthewrite-aheadloggarbage-collectionqueuesizetofallbelowthethrottlingthreshold.Ifthequeuesizedecreasesbeforethemaximumwaittimeisover,theoperationwillbeexecutednormally.Ifthequeuesizedoesnotdecreasebeforethewaittimeisover,theoperationwillbeabortedwithanerror.Thisoptiononlyhasaneffectif--wal.throttle-when-pendinghasanon-zerovalue,whichisnotthedefault.
Numberofslots
Maximumnumberofslotstobeusedinparallel:--wal.slots
Configurestheamountofwriteslotsthewrite-aheadlogcangivetowriteoperationsinparallel.Anywriteoperationwillleaseaslotandreturnittothewrite-aheadlogwhenitisfinishedwritingthedata.Aslotwillremainblockeduntilthedatainitwassynchronizedtodisk.Afterthat,aslotbecomesreusablebyfollowingoperations.Therequirednumberofslotsisthusdeterminedbytheparallelityofwriteoperationsandthedisksynchronizationspeed.Slowdisksprobablyneedhighervalues,andfastdisksmayonlyrequireavaluelowerthanthedefault.
Ignorelogfileerrors
Ignorelogfileerrorswhenopeninglogfiles:--wal.ignore-logfile-errors
Ignoresanyrecoveryerrorscausedbycorruptedlogfilesonstartup.Whensettofalse,therecoveryprocedureonstartupwillfailwithanerrorwheneveritencountersacorrupted(thatincludesonlyhalf-written)logfile.Thisisasecurityprecautiontopreventdatalossincaseofdiskerrorsetc.Whentherecoveryprocedureabortsbecauseofcorruption,anycorruptedfilescanbeinspectedandfixed(orremoved)manuallyandtheservercanberestartedafterwards.
Settingtheoptiontotruewillmaketheservercontinuewiththerecoveryprocedureevenincaseitdetectscorruptlogfileentries.Inthiscaseitwillstopatthefirstcorruptedlogfileentryandignoreallothers,whichmightcausedataloss.
Ignorerecoveryerrors
Ignorerecoveryerrors:--wal.ignore-recovery-errors
Ignoresanyrecoveryerrorsnotcausedbycorruptedlogfilesbutbylogicalerrors.Logicalerrorscanoccuriflogfilesoranyotherserverdatafileshavebeenmanuallyeditedortheserverissomehowmisconfigured.
Ignore(non-WAL)datafileerrors
Ignoredatafileerrorswhenloadingcollections:--database.ignore-datafile-errorsboolean
Ifsettofalse,CRCmismatchandothererrorsincollectiondatafileswillleadtoacollectionnotbeingloadedatall.Thecollectioninthiscasebecomesunavailable.IfsuchcollectionneedstobeloadedduringWALrecovery,theWALrecoverywillalsoabort(ifnotforcedwithoption--wal.ignore-recovery-errorstrue).
Write-AheadLogOptions
407
Settingthisflagtofalseprotectsusersfromunintentionallyusingacollectionwithcorrupteddatafiles,fromwhichonlyasubsetoftheoriginaldatacanberecovered.Workingwithsuchcollectioncouldleadtodatalossandfollowuperrors.Inordertoaccesssuchcollection,itisrequiredtoinspectandrepairthecollectiondatafilewiththedatafiledebugger(arango-dfdb).
Ifsettotrue,CRCmismatchandothererrorsduringtheloadingofacollectionwillleadtothedatafilebeingpartiallyloaded,uptothepositionofthefirsterror.Alldatauptountiltheinvalidpositionwillbeloaded.Thiswillenableuserstocontinuewithcollectiondatafileseveniftheyarecorrupted,butthiswillresultinonlyapartialloadoftheoriginaldataandpotentialfollowuperrors.TheWALrecoverywillstillabortwhenencounteringacollectionwithacorrupteddatafile,atleastif--wal.ignore-recovery-errorsisnotsettotrue.
Thedefaultvalueisfalse,socollectionswithcorrupteddatafileswillnotbeloadedatall,preventingpartialloadsandfollowuperrors.However,ifsuchcollectionisrequiredatserverstartup,duringWALrecovery,theserverwillaborttherecoveryandrefusetostart.
Write-AheadLogOptions
408
MMFilesCompactionoptionsTheArangoDBMMFilesstorageenginewillrunacompactionoverdatafiles.
ArangoDBwritesDocumentsintheWALfile.Oncetheyhavebeensealedinthewalfile,thecollectormaycopythemintoapercollectionjournalfile.
Oncejournalfilesfillup,they'resealedtobecomedatafiles.
=>onecollectionmayhavedocumentsintheWALlogs,itsjournalfile,andanarbitrarynumberofdatafiles.
Ifacollectionisloaded,eachofthesefilesareopened(thususeafilehandle)andaremmap'ed.Sincefilehandlesandmemorymappedfilesarealsoasparseresource,thatnumbershouldbekeptlow.
Onceyouupdateorremovedocumentsfromdatafiles(oralreadydidwhileitwasthejournalfile)thesedocumentsaremarkedas'dead'withadeletionmarker.
Overtimethenumberofdeaddocumentsmayrise,andwedon'twanttousethepreviouslymentionedresources,plusthediskspaceshouldbegivenbacktothesystem.Thusseveraljournalfilescanbecombinedtoone,ommittingthedeaddocuments.
Combiningseveralofthesedatafilesintooneiscalledcompaction.Thecompactionprocessreadsthealivedocumentsfromtheoriginaldatafiles,andwritesthemintonewdatafile.
Oncethatisdone,thememorymappingstotheolddatafilesisreleased,andthefilesareerased.
Sincethecompactionlocksthecollection,andalsousesI/Oresources,itscarefullyconfigurableunderwhichconditionsthesystemshouldperformwhichamountofthesecompactionjobs:
ArangoDBspawnsonecompactorthreadperdatabase.Thesettingsbelowvaryinscope.
ActivitycontrolTheactivitycontrolparametersalterthebehaviourintermsofscan/executionfrequencyofthecompaction.
Sleepintervalbetweentwocompactionruns(inseconds):--compaction.db-sleep-timeThenumberofsecondsthecollectorthreadwillwaitbetweentwoattemptstosearchforcompactabledatafilesofcollectionsinoneDatabase.Ifthecompactorhasactuallyexecutedwork,asubsequentlookupisdone.Scope:Database.
Minimumsleeptimebetweentwocompactionruns(inseconds):--compaction.min-intervalWhenanactualcompactionwasexecutedforonecollection,wewaitforthistimebeforeweexecutethecompactiononthiscollectionagain.Thisisheretoleteventuallypiledupuserloadbeworkedout.Scope:collection.
SourcedatafilesTheseparameterscontrolwhichdatafilesaretakenintoaccountforacompactionrun.Youcanspecifyseveralcriteriawhicheachoffmaybesufficcientalone.
Thescanoverthedatafilesbelongingtoonecollectionisexecutedfromoldestdatafiletonewest;iffilesqualifyforacompactiontheymaybemergedwithnewerfiles(containingyoungerdocuments)
Scope:Collectionlevel,someareinfluencedbycollectionsettings.
minimalfilesizethresholdoriginaldatafileshavetobebelowforacompaction:--compaction.min-small-data-file-sizeThisisthethresholdwhichcontrolsbelowwhichminimumtotalsizeadatafilewillalwaysbetakenintoaccountforthecompaction.
Minimumunusedcountofdocumentsinadatafile:--compaction.dead-documents-thresholdDatafileswilloftencontaindeaddocuments.Thisparameterspecifiestheirtopmostaccetpeablecountuntilthedatafilequalifiesforcompaction.
Howmanybytesofthesourcedatafileareallowedtobeunusedatmost:--compaction.dead-size-thresholdThedeaddatasizevariesalongwiththesizeofyourdocuments.Ifyouhavemanybigdocuments,thisthresholdmayhitbeforethedocumentcountthreshold.
CompactionOptions
409
Howmanypercentofthesourcedatafileshouldbeunusedatleast:--compaction.dead-size-percent-thresholdsincethesizeofthedocumentsmayvarythisthresholdworksonthepercentageofthedeaddocumentssize.Thus,ifyouhavemanyhugedeaddocuments,thisthresholdkicksinearlier.
Tonameanexamplewithnumbers,ifthedatafilecontains800kbytesofaliveand400kbytesofdeaddocuments,theshareofthedeaddocumentsis:
400/(400+800)=33%.
Ifthisvalueifhigherthanthespecifiedthreshold,thedatafilewillbecompacted.
CompactedtargetfilesOncedatafilesofacollectionarequalifiedforacompactionrun,theseparameterscontrolhowmanydatafilesaremergedintoone,(orevenonesourcedatafilemaybecompactedintoonesmallertargetdatafile)
Scope:Collectionlevel,someareinfluencedbycollectionsettings.
Maximumnumberoffilestomergetoonefile:--compaction.dest-max-filesHowmanydatafiles(atmost)wemaymergeintooneresultingdatafileduringonecompactionrun.
Howlargetheresultingfilemaybeincomparisontothecollectionsdatabase.maximal-journal-sizesetting:--compaction.dest-max-file-size-factorInArangoDByoucanconfigureadefaultjournalfilesizegloballyandoverrideitonapercollectionlevel.Thisvaluecontrolsthesizeofcollecteddatafilesrelativetotheconfiguredjournalfilesizeofthecollectioninquestion.
Afactorof3meansthatthemaximumfilesizeofthecompactedfileis3timesthesizeofthemaximumcollectionjournalfilesize.
howlargemaythecompactionresultfilebecome:--compaction.dest-max-result-file-sizenexttothefactorabove,atotallymaximumallowedfilesizeinbytesmaybespecified.Thiswilloverruleallpreviousparameters.
CompactionOptions
410
ClustersOptions
Agencyendpoint
Listofagencyendpoints:--cluster.agency-endpointendpoint
Anagencyendpointtheservercanconnectto.Theoptioncanbespecifiedmultipletimes,sotheservercanuseaclusterofagencyservers.Endpointshavethefollowingpattern:
tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryptionssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryption
AtleastoneendpointmustbespecifiedorArangoDBwillrefusetostart.ItisrecommendedtospecifyatleasttwoendpointssoArangoDBhasanalternativeendpointifoneofthembecomesunavailable.
Examples
--cluster.agency-endpointtcp://192.168.1.1:4001--cluster.agency-endpointtcp://192.168.1.2:4002...
Myaddress
Thisserver'saddress/endpoint:--cluster.my-addressendpoint
Theserver'sendpointforcluster-internalcommunication.Ifspecified,itmusthavethefollowingpattern:
tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryptionssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryption
Ifnoendpointisspecified,theserverwilllookupitsinternalendpointaddressintheagency.Ifnoendpointcanbefoundintheagencyfortheserver'sid,ArangoDBwillrefusetostart.
Examples
Listenonlyoninterfacewithaddress192.168.1.1
--cluster.my-addresstcp://192.168.1.1:8530
Listenonallipv4andipv6addresses,whichareconfiguredonport8530
--cluster.my-addressssl://[::]:8530
Myrole
Thisserver'srole:--cluster.my-role[dbserver|coordinator]
Theserver'srole.Isthisinstanceadbserver(backenddataserver)oracoordinator(frontendserverforexternalandapplicationaccess)
NodeID(deprecated)
Thisserver'sid:--cluster.my-local-infoinfo
Somelocalinformationabouttheserverinthecluster,thiscanforexamplebeanIPaddresswithaprocessIDoranystringuniquetotheserver.Specifyinginfoismandatoryonstartupiftheserverid(seebelow)isnotspecified.Eachserveroftheclustermusthaveauniquelocalinfo.Thisisignoredifmy-idbelowisspecified.
ClusterOptions
411
Thisoptionisdeprecatedandwillberemovedinafuturerelease.TheclusternodeidshavebeendroppedinfavourofoncegeneratedUUIDs.
Moreadvancedoptions(shouldgenerallyremainuntouched)
Synchroneousreplicationtiming:--cluster.synchronous-replication-timeout-factordouble
Strechorclinchtimeoutsforinternalsynchroneousreplicationmechanismbetweendbservers.Allsuchtimeoutsareaffectedbythischange.Pleasechangeonlywithintentandgreatcare.Defaultat1.0.
Systemreplicationfactor:--cluster.system-replication-factorinteger
Changedefaultreplicationfactorforsystemcollections.Defaultat2.
ClusterOptions
412
RocksDBengineoptionsRocksDBisahighlyconfigurablekey-valuestoreusedtopowerourRocksDBstorageengine.Mostoftheoptionsonthispagearepass-throughoptionstotheunderlyingRocksDBinstance,andwechangeveryfewoftheirdefaultsettings.
Dependingonthestorageengineyouhavechosentheavailabilityandthescopeoftheseoptionschanges.
Incaseyouhavechosenmmfilessomeofthefollowingoptionsapplytopersistentindexes.Incaseofrocksdbitwillapplytoalldatastoredaswellasindexes.
Pass-throughoptions--rocksdb.wal-directory
AbsolutepathfortheRocksDBWALfiles.Ifleftempty,thiswilluseasubdirectoryjournalsinsidethedatadirectory.
Writebuffers
--rocksdb.write-buffer-size
Theamountofdatatobuildupineachin-memorybuffer(backedbyalogfile)beforeclosingthebufferandqueuingittobeflushedintostandardstorage.Default:64MiB.Largervaluesmayimproveperformance,especiallyforbulkloads.
--rocksdb.max-write-buffer-number
Themaximumnumberofwritebuffersthatbuiltupinmemory.Ifthisnumberisreachedbeforethebufferscanbeflushed,writeswillbeslowedorstalled.Default:2.
--rocksdb.min-write-buffer-number-to-merge
Minimumnumberofwritebuffersthatwillbemergedtogetherwhenflushingtonormalstorage.Default:1.
--rocksdb.max-total-wal-size
MaximumtotalsizeofWALfilesthat,whenreached,willforceaflushofallcolumnfamilieswhosedataisbackedbytheoldestWALfiles.Settingthistoalowvaluewilltriggerregularflushingofcolumnfamilydatafrommemtables,sothatWALfilescanbemovedtothearchive.SettingthistoahighvaluewillavoidregularflushingbutmaypreventWALfilesfrombeingmovedtothearchiveandbeingremoved.
--rocksdb.delayed-write-rate(Hidden)
LimitedwriteratetoDB(inbytespersecond)ifwearewritingtothelastin-memorybufferallowedandweallowmorethan3buffers.Default:16MiB/s.
LSMtreestructure
--rocksdb.num-levels
ThenumberoflevelsforthedatabaseintheLSMtree.Default:7.
--rocksdb.num-uncompressed-levels
Thenumberoflevelsthatdonotusecompression.Thedefaultvalueis2.LevelsabovethisnumberwilluseSnappycompressiontoreducethediskspacerequirementsforstoringdataintheselevels.
--rocksdb.dynamic-level-bytes
Iftrue,theamountofdataineachleveloftheLSMtreeisdetermineddynamicallysoastominimizethespaceamplification;otherwise,thelevelsizesarefixed.ThedynamicsizingallowsRocksDBtomaintainawell-structuredLSMtreeregardlessoftotaldatasize.Default:true.
--rocksdb.max-bytes-for-level-base
Themaximumtotaldatasizeinbytesinlevel-1oftheLSMtree.Onlyeffectiveif--rocksdb.dynamic-level-bytesisfalse.Default:256MiB.
RocksDBEngineOptions
413
--rocksdb.max-bytes-for-level-multiplier
ThemaximumtotaldatasizeinbytesforlevelLoftheLSMtreecanbecalculatedasmax-bytes-for-level-base*(max-bytes-for-level-multiplier^(L-1)).Onlyeffectiveif--rocksdb.dynamic-level-bytesisfalse.Default:10.
--rocksdb.level0-compaction-trigger
Compactionoflevel-0tolevel-1istriggeredwhenthismanyfilesexistinlevel-0.Settingthistoahighernumbermayhelpbulkwritesattheexpenseofslowingdownreads.Default:2.
--rocksdb.level0-slowdown-trigger
Whenthismanyfilesaccumulateinlevel-0,writeswillbesloweddownto--rocksdb.delayed-write-ratetoallowcompactiontocatchup.Default:20.
--rocksdb.level0-stop-trigger
Whenthismanyfilesaccumulateinlevel-0,writeswillbestoppedtoallowcompactiontocatchup.Default:36.
FileI/O
--rocksdb.compaction-read-ahead-size
Ifnon-zero,weperformbiggerreadswhendoingcompaction.Ifyou'rerunningRocksDBonspinningdisks,youshouldsetthistoatleast2MiB.ThatwayRocksDB'scompactionisdoingsequentialinsteadofrandomreads.Default:0.
--rocksdb.use-direct-reads(Hidden)
OnlymeaningfulonLinux.Ifset,useO_DIRECTforreadingfiles.Default:false.
--rocksdb.use-direct-io-for-flush-and-compaction(Hidden)
OnlymeaningfulonLinux.Ifset,useO_DIRECTforwritingfiles.Default:false.
--rocksdb.use-fsync(Hidden)
Ifset,issueanfsynccallwhenwritingtodisk(settofalsetoissuefdatasynconly.Default:false.
Backgroundtasks
--rocksdb.max-background-jobs
Maximumnumberofconcurrentbackgroundcompactionjobs,submittedtothelowprioritythreadpool.Default:numberofprocessors.
--rocksdb.num-threads-priority-high
Numberofthreadsforhighpriorityoperations(e.g.flush).Werecommendsettingthisequaltomax-background-flushes.Default:numberofprocessors/2.
--rocksdb.num-threads-priority-low
Numberofthreadsforlowpriorityoperations(e.g.compaction).Default:numberofprocessors/2.
Caching
--rocksdb.block-cache-size
Thisisthesizeoftheblockcacheinbytes.Increasingthismayimproveperformance.Ifthereislessthan4GiBofRAMonthesystem,thedefaultvalueis256MiB.Ifthereismore,thedefaultis(systemRAMsize-2GiB)*0.3.
--rocksdb.block-cache-shard-bits
Thenumberofbitsusedtoshardtheblockcachetoallowconcurrentoperations.Tokeepindividualshardsatareasonablesize(i.e.atleast512KB),keepthisvaluetoatmostblock-cache-shard-bits/512KB.Default:block-cache-size/2^19.
--rocksdb.table-block-size
Approximatesizeofuserdata(inbytes)packedperblockforuncompresseddata.
--rocksdb.recycle-log-file-num(Hidden)
Numberoflogfilestokeeparoundforrecycling.Default:0.
RocksDBEngineOptions
414
Miscellaneous
--rocksdb.optimize-filters-for-hits(Hidden)
Thisflagspecifiesthattheimplementationshouldoptimizethefiltersmainlyforcaseswherekeysarefoundratherthanalsooptimizeforthecasewherekeysarenot.Thiswouldbeusedincaseswheretheapplicationknowsthatthereareveryfewmissesortheperformanceinthecaseofmissesisnotasimportant.Default:false.
--rocksdb.wal-recovery-skip-corrupted(Hidden)
Iftrue,skipcorruptedrecordsinWALrecovery.Default:false.
Non-Pass-ThroughOptions--rocksdb.wal-file-timeout(Hidden)
TimeoutafterwhichunusedWALfilesaredeleted(inseconds).Default:10.0s.
DataofongoingtransactionsisstoredinRAM.Transactionsthatgettoobig(intermsofnumberofoperationsinvolvedorthetotalsizeofdatacreatedormodifiedbythetransaction)willbecommittedautomatically.EffectivelythismeansthatbigusertransactionsaresplitintomultiplesmallerRocksDBtransactionsthatarecommittedindividually.TheentireusertransactionwillnotnecessarilyhaveACIDpropertiesinthiscase.
ThefollowingoptionscanbeusedtocontroltheRAMusageandautomaticintermediatecommitsfortheRocksDBengine:
--rocksdb.max-transaction-size
Transactionsizelimit(inbytes).TransactionsstoreallkeysandvaluesinRAM,solargetransactionsruntheriskofcausingout-of-memorysitations.Thissettingallowsyoutoensurethatdoesnothappenbylimitingthesizeofanyindividualtransaction.TransactionswhoseoperationswouldconsumemoreRAMthanthisthresholdvaluewillabortautomaticallywitherror32("resourcelimitexceeded").
--rocksdb.intermediate-commit-size
Ifthesizeofalloperationsinatransactionreachesthisthreshold,thetransactioniscommittedautomaticallyandanewtransactionisstarted.Thevalueisspecifiedinbytes.
--rocksdb.intermediate-commit-count
Ifthenumberofoperationsinatransactionreachesthisvalue,thetransactioniscommittedautomaticallyandanewtransactionisstarted.
--rocksdb.throttle
Ifenabled,throttlestheingestrateofwritesifnecessarytoreducechancesofcompactionsgettingtoofarbehindandblockingincomingwrites.Thisoptionistruebydefault.
RocksDBEngineOptions
415
HashcacheoptionsSinceArangoDB3.2,theseveralcorecomponentsoftheserveruseacachesystemwhichpoolsmemoryacrossmanydifferentcachetables.Inordertoprovideintelligentinternalmemorymanagement,thesystemperiodicallyreclaimsmemoryfromcacheswhichareusedlessoftenandreallocatesittocacheswhichgetmoreactivity.
Cachesize
Globalsizelimitforallhashcaches:--cache.size
Theglobalcachingsystem,allcaches,andallthedatacontainedthereinwillfitinsidethislimit.Thesizeisspecifiedinbytes.Ifthereislessthan4GiBofRAMonthesystem,thedefaultvalueis256MiB.Ifthereismore,thedefaultis(systemRAMsize-2GiB)*0.3.
Rebalancinginterval
Timebetweencacherebalancingattempts:--cache.rebalancing-interval
Thevalueisspecifiedinmicrosecondswithadefaultof2secondsandaminimumof500milliseconds.
HashCacheOptions
416
AsynchronousTasks
maximalqueuesize
Maximumsizeofthequeueforrequests:--server.maximal-queue-sizesize
Specifiesthemaximumsizeofthequeueforasynchronoustaskexecution.Ifthequeuealreadycontainssizetasks,newtaskswillberejecteduntilothertasksarepoppedfromthequeue.Settingthisvaluemayhelppreventingfromrunningoutofmemoryifthequeueisfilledupfasterthantheservercanprocessrequests.
AsynchronousTasks
417
DurabilityConfiguration
GlobalConfiguration
Thereareglobalconfigurationvaluesfordurability,whichcanbeadjustedbyspecifyingthefollowingconfigurationoptions:
defaultwaitforsyncbehavior--database.wait-for-syncbooleanDefaultwait-for-syncvalue.Canbeoverwrittenwhencreatinganewcollection.Thedefaultisfalse.
forcesyncingofcollectionpropertiestodisk--database.force-sync-propertiesbooleanForcesyncingofcollectionpropertiestodiskaftercreatingacollectionorupdatingitsproperties.Ifturnedoff,nofsyncwillhappenforthecollectionanddatabasepropertiesstoredinparameter.jsonfilesinthefilesystem.Turningoffthisoptionwillspeedupworkloadsthatcreateanddropalotofcollections(e.g.testsuites).Thedefaultistrue.
intervalforautomatic,non-requesteddisksyncs--wal.sync-intervalTheinterval(inmilliseconds)thatArangoDBwillusetoautomaticallysynchronizedatainitswrite-aheadlogstodisk.Automaticsyncswillonlybeperformedfornot-yetsynchronizeddata,andonlyforoperationsthathavebeenexecutedwithoutthewaitForSyncattribute.
Per-collectionconfiguration
Youcanalsoconfigurethedurabilitybehavioronaper-collectionbasis.UsetheArangoDBshelltochangetheseproperties.
getsorsetsthepropertiesofacollectioncollection.properties()Returnsanobjectcontainingallcollectionproperties.
waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheMMFilesstorageengineonly.isVolatile:IftruethenthecollectiondatawillbekeptinmemoryonlyandArangoDBwillnotwriteorsyncthedatatodisk.ThisoptionismeaningfulfortheMMFilesstorageengineonly.keyOptions(optional)additionaloptionsforkeygeneration.ThisisaJSONarraycontainingthefollowingattributes(note:someoftheattributesareoptional):
type:thetypeofthekeygeneratorusedforthecollection.allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeofdocumentsisconsideredanerror.increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
indexBuckets:numberofbucketsintowhichindexesusingahashtablearesplit.Thedefaultis16andthisnumberhastobeapowerof2andlessthanorequalto1024.ThisoptionismeaningfulfortheMMFilesstorageengineonly.Forverylargecollectionsoneshouldincreasethistoavoidlongpauseswhenthehashtablehastobeinitiallybuiltorresized,sincebucketsareresizedindividuallyandcanbeinitiallybuiltinparallel.Forexample,64mightbeasensiblevalueforacollectionwith100000000documents.Currently,onlytheedgeindexrespectsthisvalue,butotherindextypesmightfollowinfutureArangoDBversions.Changes(seebelow)areappliedwhenthecollectionisloadedthenexttime.Inaclustersetup,theresultwillalsocontainthefollowingattributes:numberOfShards:thenumberofshardsofthecollection.shardKeys:containsthenamesofdocumentattributesthatareusedtodeterminethetargetshardfordocuments.replicationFactor:determineshowmanycopiesofeachshardarekeptondifferentDBServers.collection.properties(properties)Changesthecollectionproperties.propertiesmustbeanobjectwithoneormoreofthefollowingattribute(s):waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheMMFilesstorageengineonly.indexBuckets:Seeabove,changesareonlyappliedwhenthecollectionisloadedthenexttime.ThisoptionismeaningfulfortheMMFilesstorageengineonly.replicationFactor:ChangethenumberofshardcopieskeptondifferentDBServers,validvaluesareintegernumbersintherangeof1-10(Clusteronly)Note:itisnotpossibletochangethejournalsizeafterthejournalordatafilehasbeencreated.Changingthisparameterwillonlyeffectnewlycreatedjournals.Alsonotethatyoucannotlowerthejournalsizetolessthensizeofthelargestdocumentalreadystoredinthecollection.Note:someothercollectionproperties,suchastype,isVolatile,orkeyOptionscannotbe
Durability
418
changedoncethecollectioniscreated.
Examples
Readallproperties
arangosh>db.example.properties();
showexecutionresultsChangeaproperty
arangosh>db.example.properties({waitForSync:true});
showexecutionresults
Per-operationconfiguration
Manydata-modificationoperationsandalsoArangoDB'stransactionsallowtospecifyawaitForSyncattribute,whichwhensetensurestheoperationdatahasbeensynchronizedtodiskwhentheoperationreturns.
Disk-UsageConfiguration
TheamountofdiskspaceusedbyArangoDBisdeterminedbyafewconfigurationoptions.
GlobalConfiguration
ThetotalamountofdiskstoragerequiredbyArangoDBisdeterminedbythesizeofthewrite-aheadlogfilesplusthesizesofthecollectionjournalsanddatafiles.
Therearethefollowingoptionsforconfiguringthenumberandsizesofthewrite-aheadlogfiles:
maximumnumberofreservelogfiles--wal.reserve-logfilesThemaximumnumberofreservelogfilesthatArangoDBwillcreateinabackgroundprocess.Reservelogfilesareusefulinthesituationwhenanoperationneedstobewrittentoalogfilebutthereservespaceinthelogfileistoolowforstoringtheoperation.Inthiscase,anewlogfileneedstobecreatedtostoretheoperation.Creatingnewlogfilesisnormallyslow,soArangoDBwilltrytopre-createlogfilesinabackgroundprocesssotherearealwaysreservelogfileswhentheactivelogfilegetsfull.ThenumberofreservelogfilesthatArangoDBkeepsinthebackgroundisconfigurablewiththisoption.
maximumnumberofhistoriclogfiles--wal.historic-logfilesThemaximumnumberofhistoriclogfilesthatArangoDBwillkeepaftertheyhavebeengarbage-collected.Ifnoreplicationisused,thereisnoneedtokeephistoriclogfilesexceptforhavingalocalchangelog.Inareplicationsetup,thenumberofhistoriclogfilesaffectstheamountofdataaslavecanfetchfromthemaster'slogs.Themorehistoriclogfiles,themorehistoricdataisavailableforaslave,whichisusefuliftheconnectionbetweenmasterandslaveisunstableorslow.Nothavingenoughhistoriclogfilesavailablemightleadtologfiledatabeingdeletedonthemasteralreadybeforeaslavehasfetchedit.
thesizeofeachWALlogfile--wal.logfile-sizeSpecifiesthefilesize(inbytes)foreachwrite-aheadlogfile.Thelogfilesizeshouldbechosensothateachlogfilecanstoreaconsiderableamountofdocuments.Thebiggerthelogfilesizeischosen,thelongeritwilltaketofillupasinglelogfile,whichalsoinfluencesthedelayuntilthedatainalogfilewillbegarbage-collectedandwrittentocollectionjournalsanddatafiles.Italsoaffectshowlonglogfilerecoverywilltakeatserverstart.
whetherornotoversizeentriesareallowed--wal.allow-oversize-entriesWhetherornotitisallowedtostoreindividualdocumentsthatarebiggerthanwouldfitintoasinglelogfile.Settingtheoptiontofalsewillmakesuchoperationsfailwithanerror.Settingtheoptiontotruewillmakesuchoperationssucceed,butwithahighpotentialperformanceimpact.Thereasonisthatforeachoversizeoperation,anindividualoversizelogfileneedstobecreatedwhichmayalsoblockotheroperations.Theoptionshouldbesettofalseifitiscertainthatdocumentswillalwayshaveasizesmallerthanasinglelogfile.Whendatagetscopiedfromthewrite-aheadlogfilesintothejournalsordatafilesofcollections,fileswillbecreatedonthecollectionlevel.Howbigthesefilesareisdeterminedbythefollowingglobalconfigurationvalue:
Durability
419
--database.maximal-journal-sizesizeMaximalsizeofjournalinbytes.Canbeoverwrittenwhencreatinganewcollection.Notethatthisalsolimitsthemaximalsizeofasingledocument.Thedefaultis32MB.
Per-collectionconfiguration
Thejournalsizecanalsobeadjustedonaper-collectionlevelusingthecollection'spropertiesmethod.
Durability
420
EncryptionThisfeatureisonlyavailableintheEnterpriseEdition.
WhenyoustoresensitivedatainyourArangoDBdatabase,youwanttoprotectthatdataunderallcircumstances.AtruntimeyouwillprotectitwithSSLtransportencryptionandstrongauthentication,butwhenthedataisalreadyondisk,youalsoneedprotection.ThatiswheretheEncryptionfeaturecomesin.
TheEncryptionfeatureofArangoDBwillencryptalldatathatArangoDBisstoringinyourdatabasebeforeitiswrittentodisk.
ThedataisencryptedwithAES-256-CTR,whichisastrongencryptionalgorithm,thatisverysuitableformulti-processorenvironments.Thismeansthatyourdataissafe,butyourdatabaseisstillfast,evenunderload.
MostmodernCPU'shavebuiltinsupportforhardwareAESencryption,whichmakesitevenfaster.
Note:TheEncryptionfeaturerequirestheRocksDBstorageengine.
Encryptionkeys
TheEncryptionfeatureofArangoDBrequiresasingle32-bytekeyperserver.Itisrecommendedtouseadifferentkeyforeachserver(whenoperatinginaclusterconfiguration).Makesuretoprotectthesekeys!
Thatmeans:
Donotwritethemtopersistentdisksoryourserver(s),alwaysstorethemonanin-memory(tmpfs)filesystem.Transportyourkeyssafelytoyourserver(s).Therearevarioustoolsformanagingsecretslikethis(e.g.vaultproject.io).Storeacopyofyourkeyofflineinasafeplace.Ifyouloseyourkey,thereisNOwaytogetyourdataback.
ConfigurationToactivateencryptionofyourdatabase,youneedtosupplyanencryptionkeytotheserver.
Makesuretopassthisoptiontheveryfirsttimeyoustartyourdatabase.Youcannotencryptadatabasethatalreadyexists.
Note:Youalsohavetoactivatetherocksdbstorageengine.
Encryptionkeystoredinfile
Passthefollowingoptiontoarangod:
$arangod\
--rocksdb.encryption-keyfile=/mytmpfs/mySecretKey\
--server.storage-engine=rocksdb
Thefile/mytmpfs/mySecretKeymustcontaintheencryptionkey.Thisfilemustbesecured,sothatonlyarangodcanaccessit.Youshouldalsoensurethatincasesome-onestealsthehardware,hewillnotbeabletoreadthefile.Forexample,byencryption/mytmpfsorcreatingain-memoryfile-systemunder/mytmpfs.
Encryptionkeygeneratedbyaprogram
Passthefollowingoptiontoarangod:
$arangod\
--rocksdb.encryption-key-generator=path-to-my-generator\
--server.storage-engine=rocksdb
Theprogrampath-to-my-generatoroutputtheencryptiononstandardoutputandexit.
Encryption
421
Creatingkeys
Theencryptionkeyfilemustcontain32bytesofrandomdata.
Youcancreateitwithacommandlinethis.
ddif=/dev/randombs=1count=32of=yourSecretKeyFile
Forsecurity,itisbesttocreatethesekeysoffline(awayfromyourdatabaseservers)anddirectlystoretheminyousecretmanagementtool.
Encryption
422
AuditingThisfeatureisavailableintheEnterpriseEdition.
Auditingallowsyoutomonitoraccesstothedatabaseindetail.Ingeneralauditlogsareoftheform
2016-01-0112:00:00|server|username|database|client-ip|authentication|text1|text2|...
Thetime-stampisinGMT.Thisallowstoeasilymatchlogentriesfromserversindifferenttimezones.
Thenameoftheserver.Youcanspecifyacustomnameonstartup.Otherwisethedefaulthostnameisused.
Theusernameisthe(authenticatedorunauthenticated)namesuppliedbytheclient.Adash-isprintedifnonamewasgivenbytheclient.
Thedatabasedescribesthedatabasethatwasaccessed.Pleasenotethattherearenodatabasecrossingqueries.Eachaccessisrestrictedtoonedatabase.
Theclient-ipdescribesthesourceoftherequest.
Theauthenticationdetailsthemethodsusedtoauthenticatetheuser.
Detailsabouttherequestsfollowintheadditionalfields.
Auditing
423
AuditConfigurationThisfeatureisavailableintheEnterpriseEdition.
Output--audit.outputoutput
Specifiesthetargetoftheauditlog.Possiblevaluesare
file://filenamewherefilenamecanberelativeorabsolute.
syslog://facilityorsyslog://facility/application-nametologintoasyslogserver.
Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputformultipletargets.
Hostname--audit.hostnamename
Thenameoftheserverusedinauditlogmessages.Bydefaultthesystemhostnameisused.
Configuration
424
AuditEventsThisfeatureisavailableintheEnterpriseEdition.
Authentication
Unknownauthenticationmethods
2016-10-0315:44:23|server1|-|database1|127.0.0.1:61525|-|unknownauthenticationmethod|/_api/version
Missingcredentials
2016-10-0315:39:49|server1|-|database1|127.0.0.1:61498|-|credentialsmissing|/_api/version
Wrongcredentials
2016-10-0315:47:26|server1|user1|database1|127.0.0.1:61528|httpbasic|credentialswrong|/_api/version
Passwordchangerequired
2016-10-0316:18:53|server1|user1|database1|127.0.0.1:62257|-|passwordchangerequired|/_api/version
JWTloginsucceeded
2016-10-0317:21:22|server1|-|database1|127.0.0.1:64214|httpjwt|user'root'authenticated|/_open/auth
Pleasenote,thattheusergivenasthirdpartistheuserthatrequestedthelogin.Ingeneral,itwillbeempty.
JWTloginfailed
2016-10-0317:21:22|server1|-|database1|127.0.0.1:64214|httpjwt|user'root'wrongcredentials|/_open/auth
Pleasenote,thattheusergivenasthirdpartistheuserthatrequestedthelogin.Ingeneral,itwillbeempty.
Authorization
Usernotauthorizedtoaccessdatabase
2016-10-0316:20:52|server1|user1|database2|127.0.0.1:62262|httpbasic|notauthorized|/_api/version
Databases
Createadatabase
2016-10-0415:33:25|server1|user1|database1|127.0.0.1:56920|httpbasic|createdatabase'database1'|ok|/_api/dat
abase
Events
425
Dropadatabase
2016-10-0415:33:25|server1|user1|database1|127.0.0.1:56920|httpbasic|deletedatabase'database1'|ok|/_api/dat
abase
Collections
Createacollection
2016-10-0517:35:57|server1|user1|database1|127.0.0.1:51294|httpbasic|createcollection'collection1'|ok|/_api
/collection
Truncateacollection
2016-10-0517:36:08|server1|user1|database1|127.0.0.1:51294|httpbasic|truncatecollection'collection1'|ok|/_a
pi/collection/collection1/truncate
Dropacollection
2016-10-0517:36:30|server1|user1|database1|127.0.0.1:51294|httpbasic|deletecollection'collection1'|ok|/_api
/collection/collection1
Indexes
Createaindex
2016-10-0518:19:40|server1|user1|database1|127.0.0.1:52467|httpbasic|createindexin'collection1'|ok|{"field
s":["a"],"sparse":false,"type":"skiplist","unique":false}|/_api/index?collection=collection1
Dropaindex
2016-10-0518:18:28|server1|user1|database1|127.0.0.1:52464|httpbasic|dropindex':44051'|ok|/_api/index/colle
ction1/44051
Documents
Readingasingledocument
2016-10-0412:27:55|server1|user1|database1|127.0.0.1:53699|httpbasic|createdocumentok|/_api/document/collecti
on1
Replacingasingledocument
2016-10-0412:28:08|server1|user1|database1|127.0.0.1:53699|httpbasic|replacedocumentok|/_api/document/collect
ion1/21456?ignoreRevs=false
Modifyingasingledocument
2016-10-0412:28:15|server1|user1|database1|127.0.0.1:53699|httpbasic|modifydocumentok|/_api/document/collecti
on1/21456?keepNull=true&ignoreRevs=false
Events
426
Deletingasingledocument
2016-10-0412:28:23|server1|user1|database1|127.0.0.1:53699|httpbasic|deletedocumentok|/_api/document/collecti
on1/21456?ignoreRevs=false
Forexample,ifsomeonestriestodeleteanon-existingdocument,itwillbeloggedas
2016-10-0412:28:26|server1|user1|database1|127.0.0.1:53699|httpbasic|deletedocumentfailed|/_api/document/coll
ection1/21456?ignoreRevs=false
Queries
2016-10-0612:12:10|server1|user1|database1|127.0.0.1:54232|httpbasic|querydocument|ok|foriincollection1r
eturni|/_api/cursor
Events
427
IntroductiontoReplicationReplicationallowsyoutoreplicatedataontoanothermachine.ItformsthebaseofalldisasterrecoveryandfailoverfeaturesArangoDBoffers.
ArangoDBoffersasynchronousandsynchronousreplicationwhichbothhavetheirprosandcons.Bothmodesmayandshouldbecombinedinarealworldscenarioandbeappliedintheusecasewheretheyexcelmost.
Wewilldescribeprosandconsofeachoftheminthefollowingsections.
Synchronousreplication
Synchronousreplicationonlyworkswithinaclusterandistypicallyusedformissioncriticaldatawhichmustbeaccessibleatalltimes.Synchronousreplicationgenerallystoresacopyofashard'sdataonanotherdbserverandkeepsitinsync.Essentially,whenstoringdataafterenablingsynchronousreplicationtheclusterwillwaitforallreplicastowriteallthedatabeforegreenlightingthewriteoperationtotheclient.Thiswillnaturallyincreasethelatencyabit,sinceonemorenetworkhopisneededforeachwrite.However,itwillenabletheclustertoimmediatelyfailovertoareplicawheneveranoutagehasbeendetected,withoutlosinganycommitteddata,andmostlywithoutevensignalinganerrorconditiontotheclient.
Synchronousreplicationisorganizedsuchthateveryshardhasaleaderandr-1followers,whererdenotedthereplicationfactor.ThenumberoffollowerscanbecontrolledusingthereplicationFactorparameterwheneveryoucreateacollection,thereplicationFactorparameteristhetotalnumberofcopiesbeingkept,thatis,itisoneplusthenumberoffollowers.
Satellitecollections
SatellitecollectionsaresynchronouslyreplicatedcollectionshavingadynamicreplicationFactor.Theywillreplicatealldatatoalldatabaseserversallowingthedatabaseserverstojoindatalocallyinsteadofdoingheavynetworkoperations.
Satellitecollectionsareanenterpriseonlyfeature.
Asynchronousreplication
InArangoDBanywriteoperationwillbeloggedtothewrite-aheadlog.WhenusingAsynchronousreplicationslaveswillconnecttoamasterandapplyalltheeventsfromtheloginthesameorderlocally.Afterthat,theywillhavethesamestateofdataasthemasterdatabase.
Replication
428
AsynchronousreplicationAsynchronousreplicationworksbyloggingeverydatamodificationonamasterandreplayingtheseeventsonanumberofslaves.
Transactionsarehonoredinreplication,i.e.transactionalwriteoperationswillbecomevisibleonslavesatomically.
Asallwriteoperationswillbeloggedtoamasterdatabase'swrite-aheadlog,thereplicationinArangoDBcurrentlycannotbeusedforwrite-scaling.ThemainpurposesofthereplicationincurrentArangoDBaretoprovideread-scalabilityand"hotbackups"ofspecificdatabases.
Itispossibletoconnectmultipleslavedatabasestothesamemasterdatabase.Slavedatabasesshouldbeusedasread-onlyinstances,andnouser-initiatedwriteoperationsshouldbecarriedoutonthem.Otherwisedataconflictsmayoccurthatcannotbesolvedautomatically,andthatwillmakethereplicationstop.
Inanasynchronousreplicationscenarioslaveswillpullchangesfromthemasterdatabase.Slavesneedtoknowtowhichmasterdatabasetheyshouldconnectto,butamasterdatabaseisnotawareoftheslavesthatreplicatefromit.Whenthenetworkconnectionbetweenthemasterdatabaseandaslavegoesdown,writeoperationsonthemastercancontinuenormally.Whenthenetworkisupagain,slavescanreconnecttothemasterdatabaseandtransfertheremainingchanges.Thiswillhappenautomaticallyprovidedslavesareconfiguredappropriately.
Replicationlag
Inthissetup,writeoperationsareappliedfirstinthemasterdatabase,andappliedintheslavedatabase(s)afterwards.
Forexample,let'sassumeawriteoperationisexecutedinthemasterdatabaseatpointintimet0.Tomakeaslavedatabaseapplythesameoperation,itmustfirstfetchthewriteoperation'sdatafrommasterdatabase'swrite-aheadlog,thenparseitandapplyitlocally.Thiswillhappenatsomepointintimeaftert0,let'ssayt1.
Thedifferencebetweent1andt0iscalledthereplicationlag,anditisunavoidableinasynchronousreplication.Theamountofreplicationlagdependsonmanyfactors,afewofwhichare:
thenetworkcapacitybetweentheslavesandthemastertheloadofthemasterandtheslavesthefrequencyinwhichslavespollthemasterforupdates
Betweent0andt1,thestateofdataonthemasterisnewerthanthestateofdataontheslave(s).Atpointintimet1,thestateofdataonthemasterandslave(s)isconsistentagain(providednonewdatamodificationshappenedonthemasterinbetween).Thus,thereplicationwillleadtoaneventuallyconsistentstateofdata.
Replicationconfiguration
Thereplicationisturnedoffbydefault.Inordertocreateamaster-slavesetup,theso-calledreplicationapplierneedstobeenabledontheslavedatabases.
Replicationisconfiguredonaper-databaselevel.Ifmultipledatabasearetobereplicated,thereplicationmustbesetupindividuallyperdatabase.
Thereplicationapplierontheslavecanbeusedtoperformaone-timesynchronizationwiththemaster(andthenstop),ortoperformanongoingreplicationofchanges.Toresumereplicationonslaverestart,theautoStartattributeofthereplicationappliermustbesettotrue.
Replicationoverhead
Asthemasterserversarelogginganywriteoperationinthewrite-ahead-loganywayreplicationdoesn'tcauseanyextraoverheadonthemaster.Howeveritwillofcoursecausesomeoverheadforthemastertoserveincomingreadrequestsoftheslaves.Returningtherequesteddataishoweveratrivialtaskforthemasterandshouldnotresultinanotableperformancedegrationinproduction.
AsynchronousReplication
429
Components
ReplicationLogger
Purpose
Thereplicationloggerwillwritealldata-modificationoperationsintothewrite-aheadlog.Thislogmaythenbereadbyclientstoreplayanydatamodificationonadifferentserver.
Checkingthestate
Toquerythecurrentstateofthelogger,usethestatecommand:
require("@arangodb/replication").logger.state();
Theresultmightlooklikethis:
{
"state":{
"running":true,
"lastLogTick":"133322013",
"totalEvents":16,
"time":"2014-07-06T12:58:11Z"
},
"server":{
"version":"2.2.0-devel",
"serverId":"40897075811372"
},
"clients":{
}
}
Therunningattributewillalwaysbetrue.InearlierversionsofArangoDBthereplicationwasoptionalandthiscouldhavebeenfalse.
ThetotalEventsattributeindicateshowmanylogeventshavebeenloggedsincethestartoftheArangoDBserver.Finally,thelastLogTickvalueindicatestheidofthelastoperationthatwaswrittentotheserver'swrite-aheadlog.Itcanbeusedtodeterminewhethernewoperationswerelogged,andisalsousedbythereplicationapplierforincrementalfetchingofdata.
Note:ThereplicationloggerstatecanalsobequeriedviatheHTTPAPI.
Toquerywhichdatarangesarestillavailableforreplicationclientstofetch,theloggerprovidesthefirstTickandtickRangesfunctions:
require("@arangodb/replication").logger.firstTick();
ThiswillreturntheminimumtickvaluethattheservercanprovidetoreplicationclientsviaitsreplicationAPIs.ThetickRangesfunctionreturnstheminimumandmaximumtickvaluesperlogfile:
require("@arangodb/replication").logger.tickRanges();
ReplicationApplier
Purpose
Thepurposeofthereplicationapplieristoreaddatafromamasterdatabase'seventlog,andapplythemlocally.Theapplierwillcheckthemasterdatabasefornewoperationsperiodically.Itwillperformanincrementalsynchronization,i.e.onlyaskingthemasterforoperationsthatoccurredafterthelastsynchronization.
AsynchronousReplication
430
Thereplicationapplierdoesnotgetnotifiedbythemasterdatabasewhenthereare"new"operationsavailable,butinsteadusesthepullprinciple.Itmightthustakesometime(theso-calledreplicationlag)beforeanoperationfromthemasterdatabasegetsshippedtoandappliedinaslavedatabase.
Thereplicationapplierofadatabaseisruninaseparatethread.Itmayencounterproblemswhenanoperationfromthemastercannotbeappliedsafely,orwhentheconnectiontothemasterdatabasegoesdown(networkoutage,masterdatabaseisdownorunavailableetc.).Inthiscase,thedatabase'sreplicationapplierthreadmightterminateitself.Itisthenuptotheadministratortofixtheproblemandrestartthedatabase'sreplicationapplier.
Ifthereplicationappliercannotconnecttothemasterdatabase,orthecommunicationfailsatsomepointduringthesynchronization,thereplicationapplierwilltrytoreconnecttothemasterdatabase.Itwillgiveupreconnectingonlyafteraconfigurableamountofconnectionattempts.
Thereplicationapplierstateisqueryableatanytimebyusingthestatecommandoftheapplier.Thiswillreturnthestateoftheapplierofthecurrentdatabase:
require("@arangodb/replication").applier.state();
Theresultmightlooklikethis:
{
"state":{
"running":true,
"lastAppliedContinuousTick":"152786205",
"lastProcessedContinuousTick":"152786205",
"lastAvailableContinuousTick":"152786205",
"progress":{
"time":"2014-07-06T13:04:57Z",
"message":"fetchingmasterlogfromoffset152786205",
"failedConnects":0
},
"totalRequests":38,
"totalFailedConnects":0,
"totalEvents":1,
"lastError":{
"errorNum":0
},
"time":"2014-07-06T13:04:57Z"
},
"server":{
"version":"2.2.0-devel",
"serverId":"210189384542896"
},
"endpoint":"tcp://master.example.org:8529",
"database":"_system"
}
Therunningattributeindicateswhetherthereplicationapplierofthecurrentdatabaseiscurrentlyrunningandpollingtheserveratendpointfornewevents.
Theprogress.failedConnectsattributeshowshowmanyfailedconnectionattemptsthereplicationappliercurrentlyhasencounteredinarow.Incontrast,thetotalFailedConnectsattributeindicateshowmanyfailedconnectionattemptstheapplierhasmadeintotal.ThetotalRequestsattributeshowshowmanyrequeststheapplierhassenttothemasterdatabaseintotal.ThetotalEventsattributeshowshowmanylogeventstheapplierhasreadfromthemaster.
Theprogress.messagesub-attributeprovidesabriefhintofwhattheappliercurrentlydoes(ifitisrunning).ThelastErrorattributealsohasanoptionalerrorMessagesub-attribute,showingthelatesterrormessage.TheerrorNumsub-attributeofthelastErrorattributecanbeusedbyclientstoprogrammaticallycheckforerrors.Itshouldbe0ifthereisnoerror,anditshouldbenon-zeroiftheapplierterminateditselfduetoaproblem.
Hereisanexampleofthestateafterthereplicationapplierterminateditselfdueto(repeated)connectionproblems:
{
"state":{
"running":false,
"progress":{
"time":"2014-07-06T13:14:37Z",
AsynchronousReplication
431
"message":"applierstopped",
"failedConnects":6
},
"totalRequests":79,
"totalFailedConnects":11,
"totalEvents":0,
"lastError":{
"time":"2014-07-06T13:09:41Z",
"errorMessage":"couldnotconnecttomasterattcp://master.example.org:8529:Couldnotconnectto'tcp:/...",
"errorNum":1400
},
...
}
}
Note:thestateofadatabase'sreplicationapplierisqueryableviatheHTTPAPI,too.PleaserefertoHTTPInterfaceforReplicationformoredetails.
All-in-onesetup
Tocopytheinitialdatafromtheslavetothemasterandstartthecontinuousreplication,thereisanall-in-onecommandsetupReplication:
require("@arangodb/replication").setupReplication(configuration);
Thefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthe_systemdatabase.Notethatitshouldberunontheslaveandnotthemaster:
db._useDatabase("_system");
require("@arangodb/replication").setupReplication({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
verbose:false,
includeSystem:false,
incremental:true,
autoResync:true
});
Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationisstarted,orincasetheinitialsynchronizationhasfailed.
Iftheinitialsynchronizationissuccessful,thecommandwillstorethegivenconfigurationontheslave.Italsoconfiguresthecontinuousreplicationtostartautomaticallyiftheslaveisrestarted,i.e.autoStartissettotrue.
Ifthecommandisrunwhiletheslave'sreplicationapplierisalreadyrunning,itwillfirststoptherunningapplier,dropitsconfigurationanddoaresynchronizationofdatawiththemaster.Itwillthenusetheprovidedconfigration,overwritinganypreviouslyexistingreplicationconfigurationontheslave.
StartingandStopping
Tomanuallystartandstoptheapplierinthecurrentdatabase,thestartandstopcommandscanbeusedlikethis:
require("@arangodb/replication").applier.start(<tick>);
require("@arangodb/replication").applier.stop();
Note:Startingareplicationapplierwithoutsettingupaninitialconfigurationwillfail.ThereplicationapplierwilllookforitsconfigurationinafilenamedREPLICATION-APPLIER-CONFIGinthecurrentdatabase'sdirectory.Ifthefileisnotpresent,ArangoDBwillusesomedefaultconfiguration,butitcannotguesstheendpoint(theaddressofthemasterdatabase)theappliershouldconnectto.Thusstartingtheapplierwithoutconfigurationwillfail.
Notethatatthefirsttimeyoustarttheapplier,youshouldpassthevaluereturnedinthelastLogTickattributeoftheinitialsyncoperation.
AsynchronousReplication
432
Note:Startingadatabase'sreplicationapplierviathestartcommandwillnotnecessarilystarttheapplieronthenextandfollowingArangoDBserverrestarts.Additionally,stoppingadatabase'sreplicationappliermanuallywillnotnecessarilypreventtheapplierfrombeingstartedagainonthenextserverstart.Allofthisisconfigurableseparately(hangonreading).
Note:whenstoppingandrestartingthereplicationapplierofdatabase,itwillresumewhereitlaststopped.Thisissensiblebecausereplicationlogeventsshouldbeappliedincrementally.Ifthereplicationapplierofadatabasehasneverbeenstartedbefore,itneedssometickvaluefromthemaster'slogfromwhichtostartfetchingevents.
Thereisonecaveattoconsiderwhenstoppingareplicationontheslave:iftherearestillongoingreplicatedtransactionsthatareneithercommittedoraborted,stoppingthereplicationapplierwillcausetheseoperationstobelostfortheslave.Ifthesetransactionscommitonthemasterlaterandthereplicationisresumed,theslavewillnotbeabletocommitthesetransactions,too.Thusstoppingthereplicationapplierontheslavemanuallyshouldonlybedoneifthereiscertaintythattherearenoongoingtransactionsonthemaster.
Configuration
Toconfigurethereplicationapplierofaspecificdatabase,usethepropertiescommand.Usingitwithoutanyargumentswillreturntheapplier'scurrentconfiguration:
require("@arangodb/replication").applier.properties();
Theresultmightlooklikethis:
{
"requestTimeout":600,
"connectTimeout":10,
"ignoreErrors":0,
"maxConnectRetries":10,
"chunkSize":0,
"autoStart":false,
"adaptivePolling":true,
"includeSystem":true,
"requireFromPresent":false,
"autoResync":false,
"autoResyncRetries":2,
"verbose":false
}
Note:Thereisnoendpointattributeconfiguredyet.Theendpointattributeisrequiredforthereplicationappliertobestartable.Youmayalsowanttoconfigureausernameandpasswordfortheconnectionviatheusernameandpasswordattributes.
require("@arangodb/replication").applier.properties({
endpoint:"tcp://master.domain.org:8529",
username:"root",
password:"secret",
verbose:false
});
Thiswillre-configurethereplicationapplierforthecurrentdatabase.Theconfigurationwillbeusedfromthenextstartofthereplicationapplier.Thereplicationappliercannotbere-configuredwhileitisrunning.Itmustbestoppedfirsttobere-configured.
TomakethereplicationapplierofthecurrentdatabasestartautomaticallywhentheArangoDBserverstarts,usetheautoStartattribute.
SettingtheadaptivePollingattributetotruewillmakethereplicationapplierpollthemasterdatabaseforchangeswithavariablefrequency.Thereplicationapplierwillthenlowerthefrequencywhenthemasterisidle,andincreaseitwhenthemastercanprovidenewevents).Otherwisethereplicationapplierwillpollthemasterdatabaseforchangeswithaconstantfrequency.
TheidleMinWaitTimeattributecontrolstheminimumwaittime(inseconds)thatthereplicationapplierwillintentionallyidlebeforefetchingmorelogdatafromthemasterincasethemasterhasalreadysentallitslogdata.ThiswaittimecanbeusedtocontrolthefrequencywithwhichthereplicationappliersendsHTTPlogfetchrequeststothemasterincasethereisnowriteactivityonthemaster.
TheidleMaxWaitTimeattributecontrolsthemaximumwaittime(inseconds)thatthereplicationapplierwillintentionallyidlebeforefetchingmorelogdatafromthemasterincasethemasterhasalreadysentallitslogdataandtherehavebeenpreviouslogfetchattemptsthatresultedinnomorelogdata.Thiswaittimecanbeusedtocontrolthemaximumfrequencywithwhichthereplicationappliersends
AsynchronousReplication
433
HTTPlogfetchrequeststothemasterincasethereisnowriteactivityonthemasterforlongerperiods.NotethatthisconfigurationvaluewillonlybeusediftheoptionadaptivePollingissettotrue.
Tosetatimeoutforconnectionandfollowingrequestattempts,usetheconnectTimeoutandrequestTimeoutvalues.ThemaxConnectRetriesattributeconfiguresafterhowmanyfailedconnectionattemptsinarowthereplicationapplierwillgiveupandturnitselfoff.Youmaywanttosetthistoahighvaluesothattemporarynetworkoutagesdonotleadtothereplicationapplierstoppingitself.TheconnectRetryWaitTimeattributeconfigureshowlongthereplicationapplierwillwaitbeforeretryingtheconnectiontothemasterincaseofconnectionproblems.
ThechunkSizeattributecanbeusedtocontroltheapproximatemaximumsizeofamaster'sresponse(inbytes).Settingittoalowvaluemaymakethemasterrespondfaster(lessdataisassembledbeforethemastersendstheresponse),butmayrequiremorerequest-responseroundtrips.Setitto0touseArangoDB'sbuilt-indefaultvalue.
TheincludeSystemattributecontrolswhetherchangestosystemcollections(suchas_graphsor_users)shouldbeapplied.Ifsettotrue,changesinthesecollectionswillbereplicated,otherwise,theywillnotbereplicated.Itisoftennotnecessarytoreplicatedatafromsystemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausetheslaveneedstohaveitsownsystemcollectionsinordertostartandkeepoperational.
TherequireFromPresentattributecontrolswhethertheapplierwillstartsynchronizingincaseitdetectsthatthemastercannotprovidedatafortheinitialtickvalueprovidedbytheslave.ThismaybethecaseifthemasterdoesnothaveabigenoughbacklogofhistoricWALlogfiles,andwhenthereplicationisre-startedafteralongerpause.WhenrequireFromPresentissettotrue,thenthereplicationapplierwillcheckatstartwhetherthestarttickfromwhichitstartsorresumesreplicationisstillpresentonthemaster.Ifnot,thentherewouldbedataloss.IfrequireFromPresentistrue,thereplicationapplierwillabortwithanappropriateerrormessage.Ifsettofalse,thenthereplicationapplierwillstillstart,andignorethedataloss.
TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentandautoResyncaresettotrueandthemastercannotprovidethelogdatatheslaverequests,thereplicationapplierwillstopasusual.ButduetothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,thereplicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullre-synchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.
Automaticre-synchronizationmaytransferalotofdatafromthemastertotheslaveandcanbeexpensive.Itisthereforeturnedoffbydefault.Whenturnedoff,theslavewillneverperformanautomaticre-synchronizationwiththemaster.
TheautoResyncRetriesoptioncanbeusedtocontrolthenumberofresynchronizationretriesthatwillbeperformedinarowwhenautomaticresynchronizationisenabledandkicksin.Settingthisto0willeffectivelydisableautoResync.Settingittosomeothervaluewilllimitthenumberofretriesthatareperformed.Thishelpspreventingendlessretriesincaseresynchronizationsalwaysfail.
Theverboseattributecontrolstheverbosityofthereplicationlogger.Settingittotruewillmakethereplicationapplierwritealinetothelogforeveryoperationitperforms.Thisshouldonlybeusedfordiagnosingreplicationproblems.
Thefollowingexamplewillsetmostofthediscussedpropertiesforthecurrentdatabase'sapplier:
require("@arangodb/replication").applier.properties({
endpoint:"tcp://master.domain.org:8529",
username:"root",
password:"secret",
adaptivePolling:true,
connectTimeout:15,
maxConnectRetries:100,
chunkSize:262144,
autoStart:true,
includeSystem:true,
autoResync:true,
autoResyncRetries:2,
});
Aftertheapplierisnowfullyconfigured,itcouldtheoreticallybestarted.However,wemayfirstneedaninitialsynchronizationofallcollectionsandtheirdatafromthemasterbeforewestartthereplicationapplier.
Theonlysafemethodfordoingafullsynchronization(orre-synchronization)isthusto
stopthereplicationapplierontheslave(ifcurrentlyrunning)performaninitialfullsyncwiththemasterdatabasenotethemasterdatabase'slastLogTickvalueand
AsynchronousReplication
434
startthecontinuousreplicationapplierontheslaveusingthistickvalue.
Theinitialsynchronizationforthecurrentdatabaseisexecutedwiththesynccommand:
require("@arangodb/replication").sync({
endpoint:"tcp://master.domain.org:8529",
username:"root",
password:"secret,
includeSystem:true
});
TheincludeSystemoptioncontrolswhetherdatafromsystemcollections(suchas_graphsand_users)shallbesynchronized.
TheinitialsynchronizationcanoptionallybeconfiguredtoincludeorexcludespecificcollectionsusingtherestrictTypeandrestrictCollectionparameters.
Thefollowingcommandonlysynchronizescollectionfooandbar:
require("@arangodb/replication").sync({
endpoint:"tcp://master.domain.org:8529",
username:"root",
password:"secret,
restrictType:"include",
restrictCollections:["foo","bar"]
});
UsingarestrictTypeofexclude,allcollectionsbutthespecifiedwillbesynchronized.
Warning:syncwilldoafullsynchronizationofthecollectionsinthecurrentdatabasewithcollectionspresentinthemasterdatabase.Anylocalinstancesofthecollectionsandalltheirdataareremoved!Onlyexecutethiscommandifyouaresureyouwanttoremovethelocaldata!
Assyncdoesafullsynchronization,itmighttakeawhiletoexecute.Whensynccompletessuccessfully,itreturnsanarrayofcollectionsithassynchronizedinitscollectionsattribute.Itwillalsoreturnthemasterdatabase'slastlogtickvalueatthetimethesyncwasstartedonthemaster.ThetickvalueiscontainedinthelastLogTickattributeofthesynccommand:
{
"lastLogTick":"231848833079705",
"collections":[...]
}
Nowyoucanstartthecontinuoussynchronizationforthecurrentdatabaseontheslavewiththecommand
require("@arangodb/replication").applier.start("231848833079705");
Note:Thetickvaluesshouldbetreatedasstrings.Usingnumericdatatypesfortickvaluesisunsafebecausetheymightexceedthe32bitvalueandtheIEEE754doubleaccuracyranges.
AsynchronousReplication
435
Per-DatabaseSetupThispagedescribesthereplicationprocessbasedonaspecificdatabasewithinanArangoDBinstance.Thatmeansthatonlythespecifieddatabasewillbereplicated.
Settingupaworkingmaster-slavereplicationrequirestwoArangoDBinstances:
master:thisistheinstancethatalldata-modificationoperationsshouldbedirectedtoslave:onthisinstance,we'llstartareplicationapplier,andthiswillfetchdatafromthemasterdatabase'swrite-aheadlogandapplyitsoperationslocally
Forthefollowingexamplesetup,we'llusetheinstancetcp://master.domain.org:8529asthemaster,andtheinstancetcp://slave.domain.org:8530asaslave.
Thegoalistohavealldatafromthedatabase_systemonmastertcp://master.domain.org:8529bereplicatedtothedatabase_systemontheslavetcp://slave.domain.org:8530.
Onthemaster,nothingspecialneedstobedone,asallwriteoperationswillautomaticallybeloggedinthemaster'swrite-aheadlog(WAL).
All-in-onesetup
Tomakethereplicationcopytheinitialdatafromthemastertotheslaveandstartthecontinuousreplicationontheslave,thereisanall-in-onecommand:
require("@arangodb/replication").setupReplication(configuration);
Thefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthe_systemdatabase.Notethatitshouldberunontheslaveandnotthemaster:
db._useDatabase("_system");
require("@arangodb/replication").setupReplication({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
verbose:false,
includeSystem:false,
incremental:true,
autoResync:true
});
Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationhasbeenstarted,orincasetheinitialsynchronizationhasfailed.
Iftheinitialsynchronizationissuccessful,thecommandwillstorethegivenconfigurationontheslave.Italsoconfiguresthecontinuousreplicationtostartautomaticallyiftheslaveisrestarted,i.e.autoStartissettotrue.
Ifthecommandisrunwhiletheslave'sreplicationapplierisalreadyrunning,itwillfirststoptherunningapplier,dropitsconfigurationanddoaresynchronizationofdatawiththemaster.Itwillthenusetheprovidedconfigration,overwritinganypreviouslyexistingreplicationconfigurationontheslave.
Initialsynchronization
Theinitialsynchronizationandcontinuousreplicationappliercanalsobestartedseparately.Tostartreplicationontheslave,makesuretherecurrentlyisnoreplicationapplierrunning.
Thefollowingcommandsstoparunningapplierintheslave's_systemdatabase:
db._useDatabase("_system");
AsynchronousReplication
436
require("@arangodb/replication").applier.stop();
Thestopoperationwillterminateanyreplicationactivityinthe_systemdatabaseontheslave.
Afterthat,theinitialsynchronizationcanberun.Itwillcopythecollectionsfromthemastertotheslave,overwritingexistingdata.Toruntheinitialsynchronization,executethefollowingcommandsontheslave:
db._useDatabase("_system");
require("@arangodb/replication").sync({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
verbose:false
});
Usernameandpasswordonlyneedtobespecifiedwhenthemasterrequiresauthentication.Tocheckwhatthesynchronizationiscurrentlydoing,supplysettheverboseoptiontotrue.Ifset,thesynchronizationwillcreatelogmessageswiththecurrentsynchronizationstatus.
Warning:Thesynccommandwillreplacedataintheslavedatabasewithdatafromthemasterdatabase!Onlyexecutethesecommandsifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!
ThesyncoperationwillreturnanattributenamedlastLogTickwhichwe'llneedtonote.Thelastlogtickwillbeusedasthestartingpointforsubsequentreplicationactivity.Let'sassumewegotthefollowinglastlogtick:
{
"lastLogTick":"40694126",
...
}
InitialsynchronizationfromtheArangoShell
Theinitialsynchronizationviathesynccommandmaytakealongtimetocomplete.Theshellwillblockuntiltheslavehascompletedtheinitialsynchronizationoruntilanerroroccurs.Bydefault,thesynccommandintheArangoShellwillpolltheslaveforastatusupdateevery10seconds.
Optionallythesynccommandcanbemadenon-blockingbysettingitsasyncoptiontotrue.Inthiscase,thesynccommandwillreturninstantlywithanidstring,andtheinitialsynchronizationwillrundetachedonthemaster.TofetchthecurrentstatusofthesyncprogressfromtheArangoShell,thegetSyncResultfunctioncanbeusedasfollows:
db._useDatabase("_system");
varreplication=require("@arangodb/replication");
/*runcommandinasyncmode*/
varid=replication.sync({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
async:true
});
/*nowquerythestatusofouroperation*/
print(replication.getSyncResult(id));
getSyncResultwillreturnfalseaslongasthesynchronizationisnotcomplete,andreturnthesynchronizationresultotherwise.
Continuoussynchronization
Whentheinitialsynchronizationisfinished,thecontinuousreplicationappliercanbestartedusingthelastlogtickprovidedbythesynccommand.Beforestartingit,thereisatleastoneconfigurationoptiontoconsider:replicationontheslavewillberunninguntiltheslavegetsshutdown.Whentheslaveservergetsrestarted,replicationwillbeturnedoffagain.Tochangethis,wefirstneedtoconfiguretheslave'sreplicationapplierandsetitsautoStartattribute.
AsynchronousReplication
437
Here'sthecommandtoconfigurethereplicationapplierwithseveraloptions,includingtheautoStartattribute:
db._useDatabase("_system");
require("@arangodb/replication").applier.properties({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
autoStart:true,
autoResync:true,
autoResyncRetries:2,
adaptivePolling:true,
includeSystem:false,
requireFromPresent:false,
idleMinWaitTime:0.5,
idleMaxWaitTime:1.5,
verbose:false
});
Animportantconsiderationforreplicationiswhetherdatafromsystemcollections(suchas_graphsor_users)shouldbeapplied.TheincludeSystemoptioncontrolsthat.Ifsettotrue,changesinsystemcollectionswillbereplicated.Otherwise,theywillnotbereplicated.Itisoftennotnecessarytoreplicatedatafromsystemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausetheslaveneedstohaveitsownsystemcollectionsinordertostartandkeepoperational.
TherequireFromPresentattributecontrolswhethertheapplierwillstartsynchronizingincaseitdetectsthatthemastercannotprovidedatafortheinitialtickvalueprovidedbytheslave.ThismaybethecaseifthemasterdoesnothaveabigenoughbacklogofhistoricWALlogfiles,andwhenthereplicationisre-startedafteralongerpause.WhenrequireFromPresentissettotrue,thenthereplicationapplierwillcheckatstartwhetherthestarttickfromwhichitstartsorresumesreplicationisstillpresentonthemaster.Ifnot,thentherewouldbedataloss.IfrequireFromPresentistrue,thereplicationapplierwillabortwithanappropriateerrormessage.Ifsettofalse,thenthereplicationapplierwillstillstart,andignorethedataloss.
TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentandautoResyncaresettotrueandthemastercannotprovidethelogdatatheslavehadrequested,thereplicationapplierwillstopasusual.ButduetothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,thereplicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullre-synchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.
Notethatautomaticre-synchronization(autoResyncoptionsettotrue)maytransferalotofdatafromthemastertotheslaveandcanthereforebeexpensive.Stillit'sturnedonheresothere'slessneedformanualintervention.
TheautoResyncRetriesoptioncanbeusedtocontrolthenumberofresynchronizationretriesthatwillbeperformedinarowwhenautomaticresynchronizationisenabledandkicksin.Settingthisto0willeffectivelydisableautoResync.Settingittosomeothervaluewilllimitthenumberofretriesthatareperformed.Thishelpspreventingendlessretriesincaseresynchronizationsalwaysfail.
Nowit'stimetostartthereplicationapplierontheslaveusingthelastlogtickwegotbefore:
db._useDatabase("_system");
require("@arangodb/replication").applier.start("40694126");
Thiswillreplicatealloperationshappeninginthemaster'ssystemdatabaseandapplythemontheslave,too.
Afterthat,youshouldbeabletomonitorthestateandprogressofthereplicationapplierbyexecutingthestatecommandontheslaveserver:
db._useDatabase("_system");
require("@arangodb/replication").applier.state();
Pleasenotethatstoppingthereplicationapplierontheslaveusingthestopcommandshouldbeavoided.Thereasonisthatcurrentlyongoingtransactions(thathavepartlybeenreplicatedtotheslave)willbeneedtoberestartedafterarestartofthereplicationapplier.Stoppingandrestartingthereplicationapplierontheslaveshouldthusonlybeperformedifthereiscertaintythatthemasteriscurrentlyfullyidleandalltransactionshavebeenreplicatedfully.
Notethatwhileaslavehasonlypartlyexecutedatransactionfromthemaster,itmightkeepawritelockonthecollectionsinvolvedinthetransaction.
AsynchronousReplication
438
YoumayalsowanttocheckthemasterandslavestatesviatheHTTPAPIs(seeHTTPInterfaceforReplication).
AsynchronousReplication
439
Server-levelSetupThispagedescribesthereplicationprocessbasedonacompleteArangoDBinstance.Thatmeansthatallincludeddatabaseswillbereplicated.
Settingupaworkingmaster-slavereplicationrequirestwoArangoDBinstances:
master:thisistheinstancethatalldata-modificationoperationsshouldbedirectedtoslave:onthisinstance,we'llstartareplicationapplier,andthiswillfetchdatafromthemasterdatabase'swrite-aheadlogandapplyitsoperationslocally
Forthefollowingexamplesetup,we'llusetheinstancetcp://master.domain.org:8529asthemaster,andtheinstancetcp://slave.domain.org:8530asaslave.
Thegoalistohavealldataofalldatabasesonmastertcp://master.domain.org:8529bereplicatedtotheslaveinstancetcp://slave.domain.org:8530.
Onthemaster,nothingspecialneedstobedone,asallwriteoperationswillautomaticallybeloggedinthemaster'swrite-aheadlog(WAL).
All-in-onesetup
Tomakethereplicationcopytheinitialdatafromthemastertotheslaveandstartthecontinuousreplicationontheslave,thereisanall-in-onecommand:
require("@arangodb/replication").setupReplicationGlobal(configuration);
ThefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthecompleteArangoDBinstance.Notethatitshouldberunontheslaveandnotthemaster:
db._useDatabase("_system");
require("@arangodb/replication").setupReplicationGlobal({
endpoint:"tcp://127.0.0.1:8529",
username:"root",
password:"",
autoStart:true
});
Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationhasbeenstarted,orincasetheinitialsynchronizationhasfailed.
Iftheinitialsynchronizationissuccessful,thecommandwillstorethegivenconfigurationontheslave.Italsoconfiguresthecontinuousreplicationtostartautomaticallyiftheslaveisrestarted,i.e.autoStartissettotrue.
Ifthecommandisrunwhiletheslave'sreplicationapplierisalreadyrunning,itwillfirststoptherunningapplier,dropitsconfigurationanddoaresynchronizationofdatawiththemaster.Itwillthenusetheprovidedconfigration,overwritinganypreviouslyexistingreplicationconfigurationontheslave.
Stoppingsynchronization
Theinitialsynchronizationandcontinuousreplicationappliercanalsobestartedseparately.Tostartreplicationontheslave,makesuretherecurrentlyisnoreplicationapplierrunning.
Thefollowingcommandsstoparunningapplierintheslave'sinstance:
db._useDatabase("_system");
require("@arangodb/replication").globalApplier.stop();
AsynchronousReplication
440
ThestopoperationwillterminateanyreplicationactivityintheArangoDBinstanceontheslave.
Afterthat,theinitialsynchronizationcanberun.Itwillcopythecollectionsfromthemastertotheslave,overwritingexistingdata.Toruntheinitialsynchronization,executethefollowingcommandsontheslave:
db._useDatabase("_system");
require("@arangodb/replication").syncGlobal({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
verbose:false
});
Usernameandpasswordonlyneedtobespecifiedwhenthemasterrequiresauthentication.Tocheckwhatthesynchronizationiscurrentlydoing,supplysettheverboseoptiontotrue.Ifset,thesynchronizationwillcreatelogmessageswiththecurrentsynchronizationstatus.
Warning:ThesyncGlobalcommandwillreplacedataintheslavedatabasewithdatafromthemasterdatabase!Onlyexecutethesecommandsifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!
ThesyncoperationwillreturnanattributenamedlastLogTickwhichwe'llneedtonote.Thelastlogtickwillbeusedasthestartingpointforsubsequentreplicationactivity.Let'sassumewegotthefollowinglastlogtick:
{
"lastLogTick":"40694126",
...
}
InitialsynchronizationfromtheArangoShellTheinitialsynchronizationviathesyncGlobalcommandmaytakealongtimetocomplete.Theshellwillblockuntiltheslavehascompletedtheinitialsynchronizationoruntilanerroroccurs.Bydefault,thesyncGlobalcommandintheArangoShellwillpolltheslaveforastatusupdateevery10seconds.
OptionallythesyncGlobalcommandcanbemadenon-blockingbysettingitsasyncoptiontotrue.Inthiscase,thesyncGlobalcommandwillreturninstantlywithanidstring,andtheinitialsynchronizationwillrundetachedonthemaster.TofetchthecurrentstatusofthesyncGlobalprogressfromtheArangoShell,thegetSyncResultfunctioncanbeusedasfollows:
db._useDatabase("_system");
varreplication=require("@arangodb/replication");
/*runcommandinasyncmode*/
varid=replication.syncGlobal({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
async:true
});
/*nowquerythestatusofouroperation*/
print(replication.getSyncResult(id));
getSyncResultwillreturnfalseaslongasthesynchronizationisnotcomplete,andreturnthesynchronizationresultotherwise.
Continuoussynchronization
Whentheinitialsynchronizationisfinished,thecontinuousreplicationappliercanbestartedusingthelastlogtickprovidedbythesyncGlobalcommand.Beforestartingit,thereisatleastoneconfigurationoptiontoconsider:replicationontheslavewillberunninguntiltheslavegetsshutdown.Whentheslaveservergetsrestarted,replicationwillbeturnedoffagain.Tochangethis,wefirstneedtoconfiguretheslave'sreplicationapplierandsetitsautoStartattribute.
Here'sthecommandtoconfigurethereplicationapplierwithseveraloptions,includingtheautoStartattribute:
AsynchronousReplication
441
db._useDatabase("_system");
require("@arangodb/replication").globalApplier.properties({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
autoStart:true,
autoResync:true,
autoResyncRetries:2,
adaptivePolling:true,
includeSystem:false,
requireFromPresent:false,
idleMinWaitTime:0.5,
idleMaxWaitTime:1.5,
verbose:false
});
Animportantconsiderationforreplicationiswhetherdatafromsystemcollections(suchas_graphsor_users)shouldbeapplied.TheincludeSystemoptioncontrolsthat.Ifsettotrue,changesinsystemcollectionswillbereplicated.Otherwise,theywillnotbereplicated.Itisoftennotnecessarytoreplicatedatafromsystemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausetheslaveneedstohaveitsownsystemcollectionsinordertostartandkeepoperational.
TherequireFromPresentattributecontrolswhethertheapplierwillstartsynchronizingincaseitdetectsthatthemastercannotprovidedatafortheinitialtickvalueprovidedbytheslave.ThismaybethecaseifthemasterdoesnothaveabigenoughbacklogofhistoricWALlogfiles,andwhenthereplicationisre-startedafteralongerpause.WhenrequireFromPresentissettotrue,thenthereplicationapplierwillcheckatstartwhetherthestarttickfromwhichitstartsorresumesreplicationisstillpresentonthemaster.Ifnot,thentherewouldbedataloss.IfrequireFromPresentistrue,thereplicationapplierwillabortwithanappropriateerrormessage.Ifsettofalse,thenthereplicationapplierwillstillstart,andignorethedataloss.
TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentandautoResyncaresettotrueandthemastercannotprovidethelogdatatheslavehadrequested,thereplicationapplierwillstopasusual.ButduetothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,thereplicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullre-synchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.
Notethatautomaticre-synchronization(autoResyncoptionsettotrue)maytransferalotofdatafromthemastertotheslaveandcanthereforebeexpensive.Stillit'sturnedonheresothere'slessneedformanualintervention.
TheautoResyncRetriesoptioncanbeusedtocontrolthenumberofresynchronizationretriesthatwillbeperformedinarowwhenautomaticresynchronizationisenabledandkicksin.Settingthisto0willeffectivelydisableautoResync.Settingittosomeothervaluewilllimitthenumberofretriesthatareperformed.Thishelpspreventingendlessretriesincaseresynchronizationsalwaysfail.
Nowit'stimetostartthereplicationapplierontheslaveusingthelastlogtickwegotbefore:
db._useDatabase("_system");
require("@arangodb/replication").globalApplier.start("40694126");
Thiswillreplicatealloperationshappeninginthemaster'ssystemdatabaseandapplythemontheslave,too.
Afterthat,youshouldbeabletomonitorthestateandprogressofthereplicationapplierbyexecutingthestatecommandontheslaveserver:
db._useDatabase("_system");
require("@arangodb/replication").globalApplier.state();
Pleasenotethatstoppingthereplicationapplierontheslaveusingthestopcommandshouldbeavoided.Thereasonisthatcurrentlyongoingtransactions(thathavepartlybeenreplicatedtotheslave)willbeneedtoberestartedafterarestartofthereplicationapplier.Stoppingandrestartingthereplicationapplierontheslaveshouldthusonlybeperformedifthereiscertaintythatthemasteriscurrentlyfullyidleandalltransactionshavebeenreplicatedfully.
Notethatwhileaslavehasonlypartlyexecutedatransactionfromthemaster,itmightkeepawritelockonthecollectionsinvolvedinthetransaction.
YoumayalsowanttocheckthemasterandslavestatesviatheHTTPAPIs(seeHTTPInterfaceforReplication).
AsynchronousReplication
442
AsynchronousReplication
443
SyncingCollectionsInordertosynchronizedataforasinglecollectionfromamastertoaslaveinstance,thereisthesyncCollectionfunction:
Itwillfetchalldocumentsofthespecifiedcollectionfromthemasterdatabaseandstoretheminthelocalinstance.Afterthesynchronization,thecollectiondataontheslavewillbeidenticaltothedataonthemaster,providednofurtherdatachangeshappenonthemaster.AnydatachangesthatareperformedonthemasterafterthesynchronizationwasstartedwillnotbecapturedbysyncCollection,butneedtobereplicatedusingtheregularreplicationappliermechanism.
Forthefollowingexamplesetup,we'llusetheinstancetcp://master.domain.org:8529asthemaster,andtheinstancetcp://slave.domain.org:8530asaslave.
Thegoalistohavealldatafromthecollectiontestindatabase_systemonmastertcp://master.domain.org:8529bereplicatedtothecollectiontestindatabase_systemontheslavetcp://slave.domain.org:8530.
Onthemaster,thecollectiontestneedstobepresentinthe_systemdatabase,withanydatainit.
Totransferthiscollectiontotheslave,issuethefollowingcommandsthere:
db._useDatabase("_system");
require("@arangodb/replication").syncCollection("test",{
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd"
});
Warning:ThesyncCollectioncommandwillreplacethecollection'sdataintheslavedatabasewithdatafromthemasterdatabase!Onlyexecutethesecommandsifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!
SettingtheoptionalincrementalattributeinthecalltosyncCollectionwillstartanincrementaltransferofdata.Thismaybeusefulincasewhentheslavealreadyhaspartsoralmostallofthedatainthecollectionandonlythedifferencesneedtobesynchronized.Notethattocomputethedifferencestheincrementaltransferwillbuildasortedlistofalldocumentkeysinthecollectiononboththeslaveandthemaster,whichmaystillbeexpensiveforhugecollectionsintermsofmemoryusageandruntime.Duringbuildingthelistofkeysthecollectionwillberead-lockedonthemaster.
TheinitialSyncMaxWaitTimeattributeinthecalltosyncCollectioncontrolshowlongtheslavewillwaitforamaster'sresponse.Thiswaittimecanbeusedtocontrolafterwhattimethesynchronizationwillgiveupandfail.
ThesyncCollectioncommandmaytakealongtimetocompleteifthecollectionisbig.Theshellwillblockuntiltheslavehassynchronizedtheentirecollectionfromthemasteroruntilanerroroccurs.Bydefault,thesyncCollectioncommandintheArangoShellwillpollforastatusupdateevery10seconds.
WhensyncCollectioniscalledfromtheArangoShell,theoptionalasyncattributecanbeusedtostartthesynchronizationasabackgroundprocessontheslave.Ifasyncissettotrue,thecalltosyncCollectionwillreturnalmostinstantlywithanidstring.Usingthisidstring,thestatusofthesyncjobontheslavecanbequeriedusingthegetSyncResultfunctionasfollows:
db._useDatabase("_system");
varreplication=require("@arangodb/replication");
/*runcommandinasyncmode*/
varid=replication.syncCollection("test",{
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
async:true
});
/*nowquerythestatusofouroperation*/
print(replication.getSyncResult(id));
getSyncResultwillreturnfalseaslongasthesynchronizationisnotcomplete,andreturnthesynchronizationresultotherwise.
AsynchronousReplication
444
AsynchronousReplication
445
ReplicationLimitationsThereplicationinArangoDBhasafewlimitations.SomeoftheselimitationsmayberemovedinlaterversionsofArangoDB:
thereisnofeedbackfromtheslavestothemaster.Ifaslavecannotapplyaneventitgotfromthemaster,themasterwillhaveadifferentstateofdata.Inthiscase,thereplicationapplierontheslavewillstopandreportanerror.Administratorscantheneither"fix"theproblemorre-syncthedatafromthemastertotheslaveandstarttheapplieragain.atthemomentitisassumedthatonlythereplicationapplierexecuteswriteoperationsonaslave.ArangoDBcurrentlydoesnotpreventusersfromcarryingouttheirownwriteoperationsonslaves,thoughthismightleadtoundefinedbehaviorandthereplicationapplierstopping.whenareplicationslaveasksamasterforlogevents,thereplicationmasterwillreturnallwriteoperationsforuser-definedcollections,butitwillexcludewriteoperationsforcertainsystemcollections.Thefollowingcollectionsareexcludedintentionallyfromreplication:_apps,_trx,_replication,_configuration,_jobs,_queues,_sessions,_foxxlogandallstatisticscollections.Writeoperationsforthefollowingsystemcollectionscanbequeriedfromamaster:_aqlfunctions,_graphs,_users.Foxxapplicationsconsistofdatabaseentriesandapplicationscriptsinthefilesystem.ThefilesystempartsofFoxxapplicationsarenottrackedanywhereandthusnotreplicatedincurrentversionsofArangoDB.ToreplicateaFoxxapplication,itisrequiredtocopytheapplicationtotheremoteserverandinstallitthereusingthefoxx-managerutility.masterserversdonotknowwhichslavesareorwillbeconnectedtothem.Allserversinareplicationsetuparecurrentlyonlylooselycoupled.Therecurrentlyisnowayforaclienttoquerywhichserversarepresentinareplication.whennotusingourmesosintegrationfailovermustbehandledbyclientsorclientAPIs.therecurrentlyisonereplicationapplierperArangoDBdatabase.Itisthusnotpossibletohaveaslaveapplyoperationsfrommultiplemastersintothesametargetdatabase.replicationissetuponaper-databaselevel.WhenusingArangoDBwithmultipledatabases,replicationmustbeconfiguredindividuallyforeachdatabase.thereplicationapplierissingle-threaded,butwriteoperationsonthemastermaybeexecutedinparalleliftheyaffectdifferentcollections.Thusthereplicationappliermightnotbeabletocatchupwithaverypowerfulandloadedmaster.replicationisonlysupportedbetweenthetwoArangoDBserversrunningthesameArangoDBversion.ItiscurrentlynotpossibletoreplicatebetweendifferentArangoDBversions.areplicationappliercannotapplydatafromitself.
AsynchronousReplication
446
SynchronousReplicationAtitscoresynchronousreplicationwillreplicatewriteoperationstomultiplehosts.ThisfeatureisonlyavailablewhenoperatingArangoDBinacluster.Wheneveracoordinatorexecutesasychronouslyreplicatedwriteoperationitwillonlybereportedtobesuccessfulifitwascarriedoutonallreplicas.IncontrasttomultimasterreplicationsetupsknownfromothersystemsArangoDB'ssynchronousoperationguaranteesaconsistentstateacrossthecluster.
SynchronousReplication
447
Implementation
Architectureinsidethecluster
SynchronousreplicationcanbeconfiguredpercollectionviathepropertyreplicationFactor.Synchronousreplicationrequiresaclustertooperate.
WheneveryouspecifyareplicationFactorgreaterthan1whencreatingacollection,synchronousreplicationwillbeactivatedforthiscollection.Theclusterwilldeterminesuitableleadersandfollowersforeveryrequestedshard(numberOfShards)withinthecluster.Whenrequestingdataofashardonlythecurrentleaderwillbeaskedwhereasfollowerswillonlykeeptheircopyinsync.Thisisduetothecurrentimplementationoftransactions.
Usingsynchronousreplicationalonewillguaranteeconsistencyandhighavailabiltyatthecostofreducedperformance:Writerequestswillhaveahigherlatency(duetoeverywrite-requesthavingtobeexecutedonthefollowers)andreadrequestswon'tscaleoutasonlytheleaderisbeingasked.
Inaclustersynchronousreplicationwillbemanagedbythecoordinatorsfortheclient.Thedatawillalwaysbestoredonprimaries.
ThefollowingexamplewillgiveyouanideaofhowsynchronousoperationhasbeenimplementedinArangoDB.
1. Connecttoacoordinatorviaarangosh2. Createacollection
127.0.0.1:8530@_system>db._create("test",{"replicationFactor":2})
3. thecoordinatorwillfigureoutaleaderand1followerandcreate1shard(asthisisthedefault)
4. Insertdata
127.0.0.1:8530@_system>db.test.insert({"replication":"ᚇ"})
5. Thecoordinatorwillwritethedatatotheleader,whichinturnwillreplicateittothefollower.
6. Onlywhenbothweresuccessfultheresultisreportedtobesuccessful
{
"_id":"test/7987",
"_key":"7987",
"_rev":"7987"
}
Whenafollowerfails,theleaderwillgiveuponitafter3secondsandproceedwiththeoperation.Assoonasthefollower(orthenetworkconnectiontotheleader)isbackup,thetwowillresynchronizeandsynchronousreplicationisresumed.Thishappensalltransparentlytotheclient.
ThecurrentimplementationofArangoDBdoesnotallowchangingthereplicationFactorlater.Thisissubjecttochange.Inthemeantimetheonlywayistodumpandrestorethecollection.Seethecookbookrecipeaboutmigrating.
Automaticfailover
Whenevertheleaderofashardisfailingandthereisaquerytryingtoaccessdataofthatshardthecoordinatorwillcontinuetryingtocontacttheleaderuntilittimeouts.Theinternalclustersupervisionrunningontheagencywillcheckclusterhealtheveryfewsecondsandwilltakeactionifthereisnoheartbeatfromaserverfor15seconds.Iftheleaderdoesn'tcomebackintimethesupervisionwillreorganizetheclusterbypromotingforeachshardafollowerthatisinsyncwithitsleadertobethenewleader.Fromthenonthecoordinatorswillcontactthenewleader.
Theprocessisbestoutlinedusinganexample:
1. Theleaderofashard(letsnameitDBServer001)isgoingdown.2. Acoordinatorisaskedtoreturnadocument:
SynchronousReplication
448
127.0.0.1:8530@_system>db.test.document("100069")
3. ThecoordinatordetermineswhichserverisresponsibleforthisdocumentandfindsDBServer001
4. ThecoordinatortriestocontactDBServer001andtimeoutsbecauseitisnotreachable.5. Afterashortwhilethesupervision(runninginparallelontheagency)willseethatheartbeatsfromDBServer001arenotcomingin6. Thesupervisionpromotesoneofthefollowers(sayDBServer002)thatisinsynctobeleaderandmakesDBServer001afollower.7. AsthecoordinatorcontinuestryingtofetchthedocumentitwillseethattheleaderchangedtoDBServer0028. Thecoordinatortriestocontactthenewleader(DBServer002)andreturnstheresult:
{
"_key":"100069",
"_id":"test/100069",
"_rev":"513",
"replication":"ᚇ"
}
9. AfterawhilethesupervisiondeclaresDBServer001tobecompletelydead.10. AnewfollowerisdeterminedfromthepoolofDBservers.11. Thenewfollowersyncsitsdatafromtheleaderandorderisrestored.
Pleasenotethattheremaystillbetimeouts.Dependingonwhenexactlytherequesthasbeendone(inregardtothesupervision)anddependingonthetimeneededtoreconfiguretheclusterthecoordinatormightfailwithatimeouterror!
SynchronousReplication
449
Configuration
Requirements
SynchronousreplicationrequiresanoperationalArangoDBcluster.
Enablingsynchronousreplication
Synchronousreplicationcanbeenabledpercollection.WhencreatingacollectionyoumayspecifythenumberofreplicasusingthereplicationFactorparameter.Thedefaultvalueissetto1whicheffectivelydisablessynchronousreplication.
Example:
127.0.0.1:8530@_system>db._create("test",{"replicationFactor":3})
Intheabovecase,anywriteoperationwillrequire2replicastoreportsuccessfromnowon.
Preparinggrowth
Youmaycreateacollectionwithhigherreplicationfactorthanavailable.Whenadditionaldbserversbecomeavailabletheshardsareautomaticallyreplicatedtothenewlyavailablemachines.
Multiplereplicasofthesameshardcannevercoexistonthesamedbserverinstance.
SynchronousReplication
450
SatelliteCollectionsSatelliteCollectionsareanEnterpriseonlyfeature.WhendoingJoinsinanArangoDBclusterdatahastoexchangedbetweendifferentservers.
Joinswillbeexecutedonacoordinator.Itwillprepareanexecutionplanandexecuteit.Whenexecutingthecoordinatorwillcontactallshardsofthestartingpointofthejoinandaskfortheirdata.Thedatabaseserverscarryingoutthisoperationwillloadalltheirlocaldataandthenasktheclusterfortheotherpartofthejoin.Thisagainwillbedistributedtoallinvolvedshardsofthisjoinpart.
Insumthisresultsinmuchnetworktrafficandslowresultsdependingoftheamountofdatathathastobesentthroughoutthecluster.
Satellitecollectionsarecollectionsthatareintendedtoaddressthisissue.
Theywillfacilitatethesynchronousreplicationandreplicateallitsdatatoalldatabaseserversthatarepartofthecluster.
ThisenablesthedatabaseserverstoexecutethatpartofanyJoinlocally.
Thisgreatlyimprovesperformanceforsuchjoinsatthecostsofincreasedstoragerequirementsandpoorerwriteperformanceonthisdata.
TocreateasatellitecollectionsetthereplicationFactorofthiscollectionto"satellite".
Usingarangosh:
arangosh>db._create("satellite",{"replicationFactor":"satellite"});
Afullexample
arangosh>varexplain=require("@arangodb/aql/explainer").explain
arangosh>db._create("satellite",{"replicationFactor":"satellite"})
arangosh>db._create("nonsatellite",{numberOfShards:8})
arangosh>db._create("nonsatellite2",{numberOfShards:8})
Let'sanalyseanormaljoinnotinvolvingsatellitecollections:
arangosh>explain("FORdocinnonsatelliteFORdoc2innonsatellite2RETURN1")
Querystring:
FORdocinnonsatelliteFORdoc2innonsatellite2RETURN1
Executionplan:
IdNodeTypeSiteEst.Comment
1SingletonNodeDBS1*ROOT
4CalculationNodeDBS1-LET#2=1/*jsonexpression*//*constassignment*/
2EnumerateCollectionNodeDBS0-FORdocINnonsatellite/*fullcollectionscan*/
12RemoteNodeCOOR0-REMOTE
13GatherNodeCOOR0-GATHER
6ScatterNodeCOOR0-SCATTER
7RemoteNodeDBS0-REMOTE
3EnumerateCollectionNodeDBS0-FORdoc2INnonsatellite2/*fullcollectionscan*/
8RemoteNodeCOOR0-REMOTE
9GatherNodeCOOR0-GATHER
5ReturnNodeCOOR0-RETURN#2
Indexesused:
none
Optimizationrulesapplied:
IdRuleName
1move-calculations-up
2scatter-in-cluster
3remove-unnecessary-remote-scatter
SatelliteCollections
451
Allshardsinvolvedqueryingthenonsatellitecollectionwillfanoutviathecoordinatortotheshardsofnonsatellite.Insum8shardswillopen8connectionstothecoordinatoraskingfortheresultsofthenonsatellite2join.Thecoordinatorwillfanouttothe8shardsofnonsatellite2.Sotherewillbequitesomenetworktraffic.
Let'snowhavealookatthesameusingsatellitecollections:
arangosh>db._query("FORdocinnonsatelliteFORdoc2insatelliteRETURN1")
Querystring:
FORdocinnonsatelliteFORdoc2insatelliteRETURN1
Executionplan:
IdNodeTypeSiteEst.Comment
1SingletonNodeDBS1*ROOT
4CalculationNodeDBS1-LET#2=1/*jsonexpression*//*constassignment*/
2EnumerateCollectionNodeDBS0-FORdocINnonsatellite/*fullcollectionscan*/
3EnumerateCollectionNodeDBS0-FORdoc2INsatellite/*fullcollectionscan,satellite*/
8RemoteNodeCOOR0-REMOTE
9GatherNodeCOOR0-GATHER
5ReturnNodeCOOR0-RETURN#2
Indexesused:
none
Optimizationrulesapplied:
IdRuleName
1move-calculations-up
2scatter-in-cluster
3remove-unnecessary-remote-scatter
4remove-satellite-joins
Inthisscenarioallshardsofnonsatellitewillbecontacted.Howeverasthejoinisasatellitejoinallshardscandothejoinlocallyasthedataisreplicatedtoallserversreducingthenetworkoverheaddramatically.
Caveats
Theclusterwillautomaticallykeepallsatellitecollectionsonallserversinsyncbyfacilitatingthesynchronousreplication.Thismeansthatwritewillbeexecutedontheleaderonlyandthisserverwillcoordinatereplicationtothefollowers.Ifafollowerdoesn'tanswerintime(duetonetworkproblems,temporaryshutdownetc.)itmayberemovedasafollower.ThisisbeingreportedtotheAgency.
Thefollower(oncebackinbusiness)willthenperiodicallychecktheAgencyandknowthatitisoutofsync.Itwillthenautomaticallycatchup.Thismaytakeawhiledependingonhowmuchdatahastobesynced.WhendoingajoininvolvingthesatelliteyoucanspecifyhowlongtheDBServerisallowedtowaitforsyncuntilthequeryisbeingaborted.
CheckAccessingCursorsfordetails.
DuringnetworkfailurethereisalsoaminimalchancethataquerywasproperlydistributedtotheDBServersbutthataprevioussatellitewritecouldnotbereplicatedtoafollowerandtheleaderdroppedthefollower.Thefollowerhoweveronlycheckseveryfewsecondsifitisreallyinsyncsoitmightindeeddeliverstaleresults.
SatelliteCollections
452
ShardingArangoDBisorganizingitscollectiondatainshards.ShardingallowstousemultiplemachinestorunaclusterofArangoDBinstancesthattogetherconstituteasingledatabase.Thisenablesyoutostoremuchmoredata,sinceArangoDBdistributesthedataautomaticallytothedifferentservers.Inmanysituationsonecanalsoreapabenefitindatathroughput,againbecausetheloadcanbedistributedtomultiplemachines.
Shardsareconfiguredpercollectionsomultipleshardsofdataformthecollectionasawhole.TodetermineinwhichshardthedataistobestoredArangoDBperformsahashacrossthevalues.Bydefaultthishashisbeingcreatedfrom_key.
Toconfigurethenumberofshards:
127.0.0.1:8529@_system>db._create("sharded_collection",{"numberOfShards":4});
Toconfigurethehashingforanotherattribute:
127.0.0.1:8529@_system>db._create("sharded_collection",{"numberOfShards":4,"shardKeys":["country"]});
Thiswouldbeusefultokeepdataofeverycountryinoneshardwhichwouldresultinbetterperformanceforqueriesworkingonapercountrybase.YoucanalsospecifymultipleshardKeys.Notehoweverthatifyouchangetheshardkeysfromtheirdefault["_key"],thenfindingadocumentinthecollectionbyitsprimarykeyinvolvesarequesttoeverysingleshard.Furthermore,inthiscaseonecannolongerprescribetheprimarykeyvalueofanewdocumentbutmustusetheautomaticallygeneratedone.Thislatterrestrictioncomesfromthefactthatensuringuniquenessoftheprimarykeywouldbeveryinefficientiftheusercouldspecifytheprimarykey.
Onwhichnodeinaclusteraparticularshardiskeptisundefined.Thereisnooptiontoconfigureanaffinitybasedoncertainshardkeys.
Uniqueindexes(hash,skiplist,persistent)onshardedcollectionsareonlyallowedifthefieldsusedtodeterminetheshardkeyarealsoincludedinthelistofattributepathsfortheindex:
shardKeys indexKeys
a a ok
a b notok
a a,b ok
a,b a notok
a,b b notok
a,b a,b ok
a,b a,b,c ok
a,b,c a,b notok
a,b,c a,b,c ok
Sharding
453
GeneralUpgradeInformation
Recommendedmajorupgradeprocedure
ToupgradeanexistingArangoDB2.xto3.0pleaseusetheproceduredescribedhere.
Recommendedminorupgradeprocedure
ToupgradeanexistingArangoDBdatabasetoanewerversionofArangoDB(e.g.3.0to3.1,or3.1to3.2),thefollowingmethodisrecommended:
ChecktheCHANGELOGandthelistofincompatiblechangesforAPIorotherchangesinthenewversionofArangoDBandmakesureyourapplicationscandealwiththemStopthe"old"arangodserviceorbinaryCopytheentire"old"datadirectorytoasafeplace(thatis,abackup)InstallthenewversionofArangoDBandstarttheserverwiththe--database.auto-upgradeoptiononce.ThismightwritetothelogfileofArangoDB,soyoumaywanttocheckthelogsforanyissuesbeforegoingon.Startthe"new"arangodserviceorbinaryregularlyandcheckthelogsforanyissues.Whenyou'reconfidenteverythingwentwell,youmaywanttocheckthedatabasedirectoryforanyfileswiththeending.old.ThesefilesarecreatedbyArangoDBduringupgradesandcanbesafelyremovedmanuallylater.
Ifanythinggoeswrongduringorshortlyaftertheupgrade:
Stopthe"new"arangodserviceorbinaryReverttothe"old"arangodbinaryandrestorethe"old"datadirectoryStartthe"old"versionagain
ItisnotsupportedtousedatafilescreatedormodifiedbyanewerversionofArangoDBwithanolderArangoDBversion.Forexample,itisunsupportedandislikelytocauseproblemswhenusing3.2datafileswithanArangoDB3.0instance.
Switchingthestorageengine
Inordertouseadifferentstorageenginewithanexistingdatadirectory,itisrequiredtofirstcreatealogicalbackupofthedatausingarangodump.
Afterthat,thearangodservershouldberestartedwiththedesiredstorageengineselected(thiscanbedonebysettingtheoption--server.storage-engine)andusinganon-existingdatadirectory.
Whentheserverisupandrunningwiththedesiredstorageengine,thedatacanbere-importedusingarangorestore.
Upgrading
454
UpgradingtoArangoDB3.3PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB3.3.Pleasebesurethatyouhavecheckedthelistofchangesin3.3beforeupgrading.
Upgradingto3.3
455
UpgradingtoArangoDB3.2PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB3.2.Pleasebesurethatyouhavecheckedthelistofchangesin3.2beforeupgrading.
Switchingthestorageengine
Inordertouseadifferentstorageenginewithanexistingdatadirectory,itisrequiredtofirstcreatealogicalbackupofthedatausingarangodump.Thatbackupshouldbecreatedbeforetheupgradeto3.2.
Afterthat,theArangoDBinstallationcanbeupgradedandstopped.Theservershouldthenberestartedwiththedesiredstorageengineselected(thiscanbedonebysettingtheoption--server.storage-engine)andusinganon-existingdatadirectory.Thiswillstarttheserverwiththeselectedstorageenginebutwithnodata.
Whentheserverisupandrunning,thedatafromthelogicalbackupcanbere-importedusingarangorestore.
Upgradingto3.2
456
UpgradingtoArangoDB3.1PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB3.1.Pleasebesurethatyouhavecheckedthelistofchangesin3.1beforeupgrading.
Upgradingto3.1
457
UpgradingtoArangoDB3.0PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB3.0.Pleasebesurethatyouhavecheckedthelistofchangesin3.0beforeupgrading.
MigratingdatabasesandcollectionsfromArangoDB2.8to3.0
ArangoDB3.0doesnotprovideanautomaticupdatemechanismfordatabasedirectoriescreatedwiththe2.xbranchesofArangoDB.
InordertomigratedatafromArangoDB2.8(oranolder2.xversion)intoArangoDB3.0,itisnecessarytoexportthedatafrom2.8usingarangodump,andthenimportthedumpintoafreshArangoDB3.0witharangorestore.
Todothis,firstrunthe2.8versionofarangodumptoexportthedatabasedataintoadirectory.arangodumpwilldumpthe_systemdatabasebydefault.Inordertomakeitdumpmultipledatabases,itneedstobeinvokedoncepersourcedatabase,e.g.
#in2.8
arangodump--server.database_system--output-directorydump-system
arangodump--server.databasemydb--output-directorydump-mydb
...
Thatwillproduceadumpdirectoryforeachdatabasethatarangodumpiscalledfor.Iftheserverhasauthenticationturnedon,itmaybenecessarytoprovidetherequiredcredentialswheninvokingarangodump,e.g.
arangodump--server.database_system--server.usernamemyuser--server.passwordmypasswd--output-directorydump-system
ThedumpsproducedbyarangodumpcannowbeimportedintoArangoDB3.0usingthe3.0versionofarangodump:
#in3.0
arangorestore--server.database_system--input-directorydump-system
arangorestore--server.databasemydb--input-directorydump-mydb
...
arangorestorewillbydefaultfailifthetargetdatabasedoesnotexist.Itcanbetoldtocreateitautomaticallyusingtheoption--create-databasetrue:
arangorestore--server.databasemydb--create-databasetrue--input-directorydump-mydb
Andagainitmayberequiredtoprovideaccesscredentialswheninvokingarangorestore:
arangorestore--server.databasemydb--create-databasetrue--server.usernamemyuser--server.passwordmypasswd--input-directo
rydump-system
Pleasenotethattheversionofdump/restoreshouldmatchtheserverversion,i.e.itisrequiredtodumptheoriginaldatawiththe2.8versionofarangodumpandrestoreitwiththe3.0versionofarangorestore.
Afterthatthe3.0instanceofArangoDBwillcontainthedatabasesandcollectionsthatwerepresentinthe2.8instance.
Adjustingauthenticationinfo
AuthenticationinformationwasstoredperdatabaseinArangoDB2.8,meaningtherecouldbedifferentusersandaccesscredentialsperdatabase.In3.0,theusersarestoredinacentrallocationinthe_systemdatabase.Tousethesameusersetupasin2.8,itmayberequiredtocreateextrausersand/oradjusttheirpermissions.
Inordertodothat,pleaseconnecttothe3.0instancewithanArangoShell(thiswillconnecttothe_systemdatabasebydefault):
arangosh--server.usernamemyuser--server.passwordmypasswd
Upgradingto3.0
458
Usethefollowingcommandstocreateanewuserwithsomepasswordandgrantthemaccesstoaspecificdatabase
require("@arangodb/users").save(username,password,true);
require("@arangodb/users").grantDatabase(username,databaseName,"rw");
Forexample,tocreateausermyuserwithpasswordmypasswdandgivethemaccesstodatabasesmydb1andmydb2,thecommandswouldlookasfollows:
require("@arangodb/users").save("myuser","mypasswd",true);
require("@arangodb/users").grantDatabase("myuser","mydb1","rw");
require("@arangodb/users").grantDatabase("myuser","mydb2","rw");
Existinguserscanalsobeupdated,removedorlistedusingthefollowingcommands:
/*updateusermyuserwithpasswordmypasswd*/
require("@arangodb/users").update("myuser","mypasswd",true);
/*removeusermyuser*/
require("@arangodb/users").remove("myuser");
/*listallusers*/
require("@arangodb/users").all();
Foxxapplications
Thedump/restoreproceduredescribedabovewillnotexportandre-importFoxxapplications.Inordertomovethesefrom2.8to3.0,Foxxapplicationsshouldbeexportedaszipfilesviathe2.8webinterface.
Thezipfilescanthenbeuploadedinthe"Services"sectionintheArangoDB3.0webinterface.Applicationsmayneedtobeadjustedmanuallytorunin3.0.PleaseconsultthemigrationguideforFoxxapps.
AnalternativewayofmovingFoxxappsinto3.0istocopythesourcedirectoryofa2.8Foxxapplicationmanuallyintothe3.0Foxxappsdirectoryforthetargetdatabase(whichisnormally/var/lib/arangodb3-apps/_db/<dbname>/buttheexactlocationisplatform-specific).
Upgradingto3.0
459
UpgradingtoArangoDB2.8PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.8.Pleasebesurethatyouhavecheckedthelistofchangesin2.8beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.8cannotbeusedwithearlierversions(e.g.ArangoDB2.7)anymore.Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDBinstallationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.8encountersadatabasecreatedwithearlierversionsofArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Databasedirectoryversion(20702)islowerthancurrentversion(208
00).
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':---------------------------------------------------------------------
-
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':--upgrade
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':---------------------------------------------------------------------
-
2015-12-04T17:11:17Z[31432]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.8startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe--upgradecommandoneachindividualdatabasesubfolder(nameddatabase-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith--upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackupofyourdatabasedirectorybeforeperformingtheupgrade.
Thelastlineoftheoutputshouldlooklikethis:
2015-12-04T17:12:15Z[31558]INFOdatabaseupgradepassed
Pleasecheckthefulloutputthe--upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.8regularly.
Upgradingaclusterplannedinthewebinterface
Upgradingto2.8
460
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellasrunningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministratedaclusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchtheclusterinthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillseeanotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillseetheusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDBinstances.Pleaseletusknowifyourunintoproblems.
ThereisanalternativewayusingtheArangoDBshell.Insteadofsteps3.and4.aboveyoucanlauncharangosh,pointittothedispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption--server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace"root"withausernameand""withapasswordthatisvalidforauthenticationwiththecluster.
UpgradingFoxxappsgeneratedbyArangoDB2.7andearlier
TheimplementationoftherequirefunctionusedtoimportmodulesinArangoDBandFoxxhaschangedinordertoimprovecompatibilitywithNode.jsmodules.
Givenanapp/servicewiththefollowinglayout:
manifest.jsoncontrollers/
todos.jsmodels/
todo.jsrepositories/
todos.jsnode_modules/
models/todo.js
Thefilecontrollers/todos.jswouldpreviouslycontainthefollowingrequirecalls:
var_=require('underscore');
varjoi=require('joi');
varFoxx=require('org/arangodb/foxx');
varArangoError=require('org/arangodb').ArangoError;
varTodos=require('repositories/todos');//<--!
varTodo=require('models/todo');//<--!
Therequirepathsrepositories/todosandmodels/todowerepreviouslyresolvedlocallyasrelativetotheapproot.
Startingwith2.8thesepathswouldinsteadberesolvedasrelativetothenode_modulesfolderortheglobalArangoDBmodulepathsbeforebeingresolvedlocallyasafallback.
Inthegivenexamplelayouttheappwouldbreakin2.8becausethemodulenamemodels/todowouldalwaysresolvetonode_modules/models/todo.js(whichpreviouslywouldhavebeenignored)insteadofthelocalmodels/todo.js.
Inordertomakesuretheappstillworksin2.8,therequirecallsincontrollers/todos.jswouldneedtobeadjustedtolooklikethis:
Upgradingto2.8
461
var_=require('underscore');
varjoi=require('joi');
varFoxx=require('org/arangodb/foxx');
varArangoError=require('org/arangodb').ArangoError;
varTodos=require('../repositories/todos');//<--!
varTodo=require('../models/todo');//<--!
Notethattheold"global"stylerequirecallsmaystillworkin2.8butmaybreakunexpectedlyifmoduleswithmatchingnamesareinstalledglobally.
Upgradingto2.8
462
UpgradingtoArangoDB2.6PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.6.Pleasebesurethatyouhavecheckedthelistofchangesin2.6beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.6cannotbeusedwithearlierversions(e.g.ArangoDB2.5)anymore.Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDBinstallationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.6encountersadatabasecreatedwithearlierversionsofArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Databasedirectoryversion(20501)islowerthancurrentversion(2060
0).
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':----------------------------------------------------------------------
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':--upgrade
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':----------------------------------------------------------------------
2015-02-17T09:43:11Z[8302]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.6startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe--upgradecommandoneachindividualdatabasesubfolder(nameddatabase-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith--upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackupofyourdatabasedirectorybeforeperformingtheupgrade.
Thelastlineoftheoutputshouldlooklikethis:
2014-12-22T12:03:31Z[12026]INFOdatabaseupgradepassed
Pleasecheckthefulloutputthe--upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.6regularly.
Upgradingaclusterplannedinthewebinterface
Upgradingto2.6
463
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellasrunningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministratedaclusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchtheclusterinthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillseeanotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillseetheusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDBinstances.Pleaseletusknowifyourunintoproblems.
ThereisanalternativewayusingtheArangoDBshell.Insteadofsteps3.and4.aboveyoucanlauncharangosh,pointittothedispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption--server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace"root"withausernameand""withapasswordthatisvalidforauthenticationwiththecluster.
Upgradingto2.6
464
UpgradingtoArangoDB2.5PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.5.Pleasebesurethatyouhavecheckedthelistofchangesin2.5beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.5cannotbeusedwithearlierversions(e.g.ArangoDB2.4)anymore.Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDBinstallationbeforeperforminganupgrade.
In2.5wehavealsochangedthepathsforFoxxapplications.PleasealsomakesurethatyouhaveabackupofallFoxxappsinyourjavascript.app-pathandjavascript.dev-app-path.ItissufficienttohavethesourcefilesforFoxxsomewhereelsesoyoucanreinstallthemonerror.Tocheckthateverythinghasworkedduringupgradeyoucouldusetheweb-interfaceApplicationstabor
unix>foxx-managerlist
forallyourdatabases.Thelistedappsshouldbeidenticalbeforeandaftertheupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.5encountersadatabasecreatedwithearlierversionsofArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Databasedirectoryversion(20401)islowerthancurrentversion(2050
0).
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':----------------------------------------------------------------------
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':--upgrade
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':----------------------------------------------------------------------
2015-02-17T09:43:11Z[8302]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.5startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption.Note:WehavechangedFoxxfolderstructureandimplementedanupgradetasktomoveyourapplicationstothenewstructure.InordertotellthisupgradetasktoalsomoveyourdevelopmentFoxxappspleasemakesureyougivethedev-app-pathaswell.IfyouhavenotuseddevelopmentmodeforFoxxappsyoucandropthe--javascript.dev-app-path.Itisonlypossibletoupgradeonedev-app-pathtogetherwithonedatafolder.
unix>arangoddata--upgrade--javascript.dev-app-pathdevapps
wheredataisArangoDB'smaindatadirectoryanddevappsisthedirectorywhereyoudevelopFoxxapps.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe--upgradecommandoneachindividualdatabasesubfolder(nameddatabase-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
Upgradingto2.5
465
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith--upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackupofyourdatabasedirectorybeforeperformingtheupgrade.
Thelastlineoftheoutputshouldlooklikethis:
2014-12-22T12:03:31Z[12026]INFOdatabaseupgradepassed
Pleasecheckthefulloutputthe--upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.5regularly.
UpgradingaclusterplannedinthewebinterfaceAclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellasrunningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministratedaclusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchtheclusterinthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillseeanotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillseetheusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDBinstances.Pleaseletusknowifyourunintoproblems.
ThereisanalternativewayusingtheArangoDBshell.Insteadofsteps3.and4.aboveyoucanlauncharangosh,pointittothedispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption--server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace"root"withausernameand""withapasswordthatisvalidforauthenticationwiththecluster.
Upgradingto2.5
466
UpgradingtoArangoDB2.4PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.4.Pleasebesurethatyouhavecheckedthelistofchangesin2.4beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.4cannotbeusedwithearlierversions(e.g.ArangoDB2.3)anymore.Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDBinstallationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.4encountersadatabasecreatedwithearlierversionsofArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':Databasedirectoryversion(20302)islowerthancurrentversion(204
00).
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':---------------------------------------------------------------------
-
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':--upgrade
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':---------------------------------------------------------------------
-
2014-12-22T12:02:28Z[12001]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.4startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption:
unix>arangoddata--upgrade
wheredataisArangoDB'smaindatadirectory.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe--upgradecommandoneachindividualdatabasesubfolder(nameddatabase-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith--upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackupofyourdatabasedirectorybeforeperformingtheupgrade.
Thelastlineoftheoutputshouldlooklikethis:
2014-12-22T12:03:31Z[12026]INFOdatabaseupgradepassed
Upgradingto2.4
467
Pleasecheckthefulloutputthe--upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.4regularly.
Upgradingaclusterplannedinthewebinterface
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellasrunningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministratedaclusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchtheclusterinthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillseeanotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillseetheusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDBinstances.Pleaseletusknowifyourunintoproblems.
ThereisanalternativewayusingtheArangoDBshell.Insteadofsteps3.and4.aboveyoucanlauncharangosh,pointittothedispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption--server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace"root"withausernameand""withapasswordthatisvalidforauthenticationwiththecluster.
Upgradingto2.4
468
UpgradingtoArangoDB2.3PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.3.Pleasebesurethatyouhavecheckedthelistofchangesin2.3beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.3cannotbeusedwithearlierversions(e.g.ArangoDB2.2)anymore.Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDBinstallationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.3encountersadatabasecreatedwithearlierversionsofArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':Databasedirectoryversion(2.2)islowerthancurrentversion(20300)
.
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':----------------------------------------------------------------------
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':--upgrade
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':----------------------------------------------------------------------
2014-11-03T15:48:06Z[2694]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.3startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption:
unix>arangoddata--upgrade
wheredataisArangoDB'smaindatadirectory.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe--upgradecommandoneachindividualdatabasesubfolder(nameddatabase-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith--upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackupofyourdatabasedirectorybeforeperformingtheupgrade.
Theoutputshouldlooklikethis:
2014-11-03T15:48:47Z[2708]INFOIndatabase'_system':Found24definedtask(s),5task(s)torun
2014-11-03T15:48:47Z[2708]INFOIndatabase'_system':stateprod/standalone/upgrade,tasksupdateUserModel,createStatistics,
upgradeClusterPlan,setupQueues,setupJobs
2014-11-03T15:48:48Z[2708]INFOIndatabase'_system':upgradesuccessfullyfinished
2014-11-03T15:48:48Z[2708]INFOdatabaseupgradepassed
Upgradingto2.3
469
Pleasechecktheoutputthe--upgraderun.Itmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifnoerrorsarepresentortheyhavebeenresolved,youcanstartArangoDB2.3regularly.
Upgradingaclusterplannedinthewebinterface
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellasrunningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministratedaclusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchtheclusterinthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillseeanotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillseetheusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDBinstances.Pleaseletusknowifyourunintoproblems.
ThereisanalternativewayusingtheArangoDBshell.Insteadofsteps3.and4.aboveyoucanlauncharangosh,pointittothedispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption--server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace"root"withausernameand""withapasswordthatisvalidforauthenticationwiththecluster.
Upgradingto2.3
470
UpgradingtoArangoDB2.2PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.2.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.2cannotbeusedwithearlierversions(e.g.ArangoDB2.1)anymore.Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDBinstallationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.2encountersadatabasecreatedwithearlierversionsofArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':Databasedirectoryversion(2.1)islowerthanserverversion(2.2).
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':---------------------------------------------------------------------
-
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':--upgrade
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':---------------------------------------------------------------------
-
2014-07-07T22:04:53Z[18675]FATALDatabaseversioncheckfailedfor'_system'.Pleasestarttheserverwiththe--upgradeopti
on
TomakeArangoDB2.2startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgradeprocedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe--upgradeoption:
unix>arangoddata--upgrade
wheredataisArangoDB'smaindatadirectory.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe--upgradecommandoneachindividualdatabasesubfolder(nameddatabase-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith--upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackupofyourdatabasedirectorybeforeperformingtheupgrade.
Theoutputshouldlooklikethis:
2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':startingupgradefromversion2.1to2.2.0
2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':Found19definedtask(s),2task(s)torun
2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':upgradesuccessfullyfinished
2014-07-07T22:11:30Z[18867]INFOdatabaseupgradepassed
Upgradingto2.2
471
Pleasechecktheoutputthe--upgraderun.Itmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifnoerrorsarepresentortheyhavebeenresolved,youcanstartArangoDB2.2regularly.
Upgradingaclusterplannedinthewebinterface
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellasrunningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministratedaclusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchtheclusterinthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillseeanotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillseetheusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDBinstances.Pleaseletusknowifyourunintoproblems.
ThereisanalternativewayusingtheArangoDBshell.Insteadofsteps3.and4.aboveyoucanlauncharangosh,pointittothedispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption--server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace"root"withausernameand""withapasswordthatisvalidforauthenticationwiththecluster.
Upgradingto2.2
472
Troubleshooting
Troubleshooting
473
ArangodIftheArangoDBserverdoesnotstartorifyoucannotconnecttoitusingarangoshorotherclients,youcantrytofindtheproblemcausebyexecutingthefollowingsteps.Iftheserverstartsupwithoutproblemsyoucanskipthissection.
Checktheserverlogfile:Iftheserverhaswrittenalogfileyoushouldcheckitbecauseitmightcontainrelevanterrorcontextinformation.
Checktheconfiguration:Theserverlooksforaconfigurationfilenamedarangod.confonstartup.Thecontentsofthisfilewillbeusedasabaseconfigurationthatcanoptionallybeoverriddenwithcommand-lineconfigurationparameters.Youshouldchecktheconfigfileforthemostrelevantparameterssuchas:
server.endpoint:WhatIPaddressandporttobindtologparameters:Ifandwheretologdatabase.directory:Paththedatabasefilesarestoredin
Iftheconfigurationrevealsthatsomethingisnotconfiguredrighttheconfigfileshouldbeadjustedandtheserverberestarted.
Starttheservermanuallyandcheckitsoutput:Startingtheservermightfailevenbeforeloggingisactivatedsotheserverwillnotproducelogoutput.Thiscanhappeniftheserverisconfiguredtowritethelogstoafilethattheserverhasnopermissionson.Inthiscasetheservercannotloganerrortothespecifiedlogfilebutwillwriteastartuperroronstderrinstead.Startingtheservermanuallywillalsoallowyoutooverridespecificconfigurationoptions,e.g.toturnon/offfileorscreenloggingetc.
ChecktheTCPport:Iftheserverstartsupbutdoesnotacceptanyincomingconnectionsthismightbeduetofirewallconfigurationbetweentheserverandanyclient(s).TheserverbydefaultwilllistenonTCPport8529.PleasemakesurethisportisactuallyaccessiblebyotherclientsifyouplantouseArangoDBinanetworksetup.
Whenusinghostnamesintheconfigurationorwhenconnecting,pleasemakesurethehostnameisactuallyresolvable.ResolvinghostnamesmightinvokeDNS,whichcanbeasourceoferrorsonitsown.
ItisgenerallygoodadvicetonotuseDNSwhenspecifyingtheendpointsandconnectionaddresses.UsingIPaddressesinsteadwillruleoutDNSasasourceoferrors.Anotheralternativeistouseahostnamespecifiedinthelocal/etc/hostsfile,whichwillthenbypassDNS.
Testifcurlcanconnect:Oncetheserverisstarted,youcanquicklyverifyifitrespondstorequestsatall.Thischeckallowsyoutodeterminewhetherconnectionerrorsareclient-specificornot.Ifatleastoneclientcanconnect,itislikelythatconnectionproblemsofotherclientsarenotduetoArangoDB'sconfigurationbutduetoclientorin-betweennetworkconfigurations.
Youcantestconnectivityusingasimplecommandsuchas:
curl--dump--XGEThttp://127.0.0.1:8529/_api/version&&echo
ThisshouldreturnaresponsewithanHTTP200statuscodewhentheserverisrunning.Ifitdoesitalsomeanstheserverisgenerallyacceptingconnections.Alternativetoolstocheckconnectivityarelynxorab.
arangod
474
EmergencyConsoleTheArangoDBdatabaseserverhastwomodesofoperation:Asaserver,whereitwillanswertoclientrequestsandasanemergencyconsole,inwhichyoucanaccessthedatabasedirectly.Thelatter-asthenamesuggests-shouldonlybeusedincaseofanemergency,forexample,acorruptedcollection.Usingtheemergencyconsoleallowsyoutoissueallcommandsnormallyavailableinactionsandtransactions.Whenstartingtheserverinemergencyconsolemode,theservercannothandleanyclientrequests.
Youshouldneverstartmorethanoneserverusingthesamedatabasedirectory,independentofthemodeofoperation.Normally,ArangoDBwillpreventyoufromdoingthisbyplacingalockfileinthedatabasedirectoryandnotallowingasecondArangoDBinstancetousethesamedatabasedirectoryifalockfileisalreadypresent.
InCaseOfDisaster
Thefollowingcommandstartsanemergencyconsole.
Note:Neverstarttheemergencyconsoleforadatabasewhichalsohasaserverattachedtoit.Ingeneral,theArangoDBshelliswhatyouwant.
>./arangod--console--logerror/tmp/vocbase
ArangoDBshell[V8version5.0.71.39,DBversion3.x.x]
arango>1+2;
3
arango>vardb=require("@arangodb").db;db.geo.count();
703
TheemergencyconsoleprovidesaJavaScriptconsoledirectlyrunninginthearangodserverprocess.ThisallowstodebugandexaminecollectionsanddocumentsaswiththenormalArangoDBshell,butwithoutclient/servercommunication.
However,itisverylikelythatyouwillneverneedtheemergencyconsoleunlessyouareanArangoDBdeveloper.
EmergencyConsole
475
DatafileDebugger
InCaseOfDisaster
AranagoDBusesappend-onlyjournals.Datacorruptionshouldonlyoccurwhenthedatabaseserveriskilled.Inthiscase,thecorruptionshouldonlyoccurinthelastobject(s)thathavebeingwrittentothejournal.
Ifacorruptionoccurswithinanormaldatafile,thenthiscanonlyhappenifahardwarefaultoccurred.
Ifajournalordatafileiscorrupt,shutdownthedatabaseserverandstarttheprogram
unix>arango-dfdb
inordertochecktheconsistencyofthedatafilesandjournals.Thisbringsup
_________________
/\___||____/_(_)|___/\/__\/_\
//\/_`|__/_`||_|||/_\//\/__\////_\/
//_//(_||||(_||_|||__///_//\/\//_\\
/___,'\__,_|\__\__,_|_||_|_|\___|/___,'\_____/\____/
Availablecollections:
0:_structures
1:_users
2:_routing
3:_modules
4:_graphs
5:products
6:prices
*:all
Collectiontocheck:
Youcannowselectwhichdatabaseandcollectionyouwanttocheck.Afteryouselectedoneorallofthecollections,aconsistencycheckwillbeperformed.
Checkingcollection#1:_users
Database
path:/usr/local/var/lib/arangodb
Collection
name:_users
identifier:82343
Datafiles
#ofjournals:1
#ofcompactors:1
#ofdatafiles:0
Datafile
path:/usr/local/var/lib/arangodb/collection-82343/journal-1065383.db
type:journal
currentsize:33554432
maximalsize:33554432
totalused:256
#ofentries:3
status:OK
Ifthereisaproblemwithoneofthedatafiles,thenthedatabasedebuggerwillprintitandpromptforwhethertoattempttofixit.
WARNING:Thejournalwasnotclosedproperly,thelastentriesarecorrupted.
ThismighthappenArangoDBwaskilledandthelastentrieswerenot
fullywrittentodisk.
DatafileDebugger
476
Wipethelastentries(Y/N)?
IfyouanswerY,thecorruptedentrywillberemoved.
Ifyouseeacorruptioninadatafile(andnotajournal),thensomethingisterriblywrong.ThesefilesareimmutableandneverchangedbyArangoDB.Acorruptioninsuchfileisanindicationofahard-diskfailure.
DatafileDebugger
477
ArangobenchArangobenchisArangoDB'sbenchmarkandtesttool.Itcanbeusedtoissuetestrequeststothedatabaseforperformanceandserverfunctiontesting.Itsupportsparallelqueryingandbatchrequests.
Relatedblogposts:
MeasuringArangoDBinsertperformanceGainfactorof5usingbatchrequests
Startupoptions--async:Sendasynchronousrequests.Thedefaultvalueisfalse.
--batch-size:Numberofoperationstosendperbatch.Use0todisablebatching(thisisthedefault).
--collection:Nameofcollectiontouseintest(onlyrelevantforteststhatinvokecollections).
--replication-factor:Incaseofacluster,thereplicationfactorofthecreatedcollections.
--number-of-shards:Incaseofacluster,thenumberofshardsofthecreatedcollections.
--wait-for-sync:ThevalueofwaitForSyncforcreatedcollections.
--complexity:Complexityvaluefortestcase(default:1).Meaningdependsontestcase.
--concurrency:Numberofparallelthreadsthatwillissuerequests(default:1).
--configuration:Readconfigurationfromfile.
--delay:Useastartupdelay.Thisisonlynecessarywhenruninseries.Thedefaultvalueisfalse.
--keep-alive:UseHTTPkeep-alive(default:true).
--progress:Showprogressofbenchmark,onevery20threquest.Settofalsetodisableintermediatelogging.Thedefaultvalueistrue.
--requests:Totalnumberofrequeststoperform(default:1000).
--server.endpoint:Serverendpointtoconnectto,consistingofprotocol,IPaddressandport.Defaultstotcp://localhost:8529.
--server.database:Databasenametousewhenconnecting(default:"_system").
--server.username:Usernametousewhenconnecting(default:"root").
--server.password:Passwordtousewhenconnecting.Don'tspecifythisoptiontogetapasswordprompt.
--server.authentication:Wetherornottoshowthepasswordpromptanduseauthenticationwhenconnectingtotheserver(default:true).
--test-case:Nameoftestcasetoperform(default:"version").Possiblevalues:
version:requests/_api/versiondocument:createsdocumentscollection:createscollectionsimport-document:createsdocumentsviatheimportAPIhash:Create/Read/Update/Readdocumentsindexedbyahashindexskiplist:Create/Read/Update/Readdocumentsindexedbyaskiplistedge:Create/Read/Updateedgedocumentsshapes:Create&Deletedocumentswithheterogeneousattributenamesshapes-append:Createdocumentswithheterogeneousattributenamesrandom-shapes:Create/Read/Deleteheterogeneousdocumentswithrandomvaluescrud:Create/Read/Update/Deletecrud-append:Create/Read/Update/Readagain
Arangobench
478
crud-write-read:Create/ReadDocumentsaqltrx:AQLTransactionswithdeepnestedAQLFOR-loopscounttrx:usesJStransactionstocountthedocumentsandinserttheresultagainmultitrx:multipletransactionscombiningreads&writesfromjsmulti-collection:multipletransactionscombiningreads&writesfromjsonmultiplecollectionsaqlinsert:insertdocumentsviaAQLaqlv8:executeAQLwithV8functionstoinsertrandomdocuments
--verbose:PrintoutrepliesiftheHTTPheaderindicatesDBerrors.(default:false).
Examples
arangobench
StartsArangobenchwiththedefaultuserandserverendpoint.
--test-caseversion--requests1000--concurrency1
Runsthe'version'testcasewith1000requests,withoutconcurrency.
--test-casedocument--requests1000--concurrency2
Runsthe'document'testcasewith2000requests,withtwoconcurrentthreads.
--test-casedocument--requests1000--concurrency2--asynctrue
Runsthe'document'testcasewith2000requests,withconcurrency2,withasyncrequests.
--test-casedocument--requests1000--concurrency2--batch-size10
Runsthe'document'testcasewith2000requests,withconcurrency2,usingbatchrequests.
Arangobench
479
Architecture
AppendOnly/MVCC
Insteadofoverwritingexistingdocuments,ArangoDBwillcreateanewversionofmodifieddocuments.Thisiseventhecasewhenadocumentgetsdeleted.Thetwobenefitsare:
Objectscanbestoredcoherentlyandcompactlyinthemainmemory.Objectsarepreserved,isolatedwritingandreadingtransactionsallowaccessingtheseobjectsforparalleloperations.
Thesystemcollectsobsoleteversionsasgarbage,recognizingthemasforsaken.Garbagecollectionisasynchronousandrunsparalleltootherprocesses.
MostlyMemory/Durability
Databasedocumentsarestoredinmemory-mappedfiles.Perdefault,thesememory-mappedfilesaresyncedregularlybutnotinstantly.Thisisoftenagoodtradeoffbetweenstorageperformanceanddurability.Ifthislevelofdurabilityistoolowforanapplication,theservercanalsosyncallmodificationstodiskinstantly.ThiswillgivefulldurabilitybutwillcomewithaperformancepenaltyaseachdatamodificationwilltriggerasyncI/Ooperation.
Architecture
480
Write-aheadlogTheWrite-aheadlogispartoftheMMFilesstorageengine;Thisdoesn'tapplytoyourArangoDBifyouarerunningwiththeRocksDBstorageengine.
Startingwithversion2.2ArangoDBstoresalldata-modificationoperationinitswrite-aheadlog.Thewrite-aheadlogissequenceofappend-onlyfilescontainingallthewriteoperationsthatwereexecutedontheserver.
Itisusedtorundatarecoveryafteraservercrash,andcanalsobeusedinareplicationsetupwhenslavesneedtoreplaythesamesequenceofoperationsasonthemaster.
Bydefault,eachwrite-aheadlogfileis32MiBinsize.Thissizeisconfigurableviatheoption--wal.logfile-size.
Whenawrite-aheadlogfileisfull,itissettoread-only,andfollowingoperationswillbewrittenintothenextwrite-aheadlogfile.Bydefault,ArangoDBwillreservesomesparelogfilesinthebackgroundsoswitchinglogfilesshouldbefast.HowmanyreservelogfilesArangoDBwilltrytokeepavailableinthebackgroundcanbecontrolledbytheconfigurationoption--wal.reserve-logfiles.
Datacontainedinfullwrite-aheadfileswilleventuallybetransferredintothejournalsordatafilesofcollections.Onlythe"surviving"documentswillbecopiedover.Whenallremainingoperationsfromawrite-aheadlogfilehavebeencopiedoverintothejournalsordatafilesofthecollections,thewrite-aheadlogfilecansafelyberemovedifitisnotusedforreplication.
Long-runningtransactionspreventwrite-aheadlogfilesfrombeingfullygarbage-collectedbecauseitisunclearwhetheratransactionwillcommitorabort.Long-runningtransactionscanthusblockthegarbage-collectionprogressandshouldthereforebeavoidedatallcosts.
Onasystemthatactsasareplicationmaster,itisusefultokeepafewofthealreadycollectedwrite-aheadlogfilessoreplicationslavesstillcanfetchdatafromthemifrequired.Howmanycollectedlogfileswillbekeptbeforetheygetdeletedisconfigurableviatheoption--wal.historic-logfiles.
Forallwrite-aheadlogconfigurationoptions,pleaserefertothepageWrite-aheadlogoptions.
Write-aheadlog
481
StorageEnginesAttheverybottomoftheArangoDBdatabaseliesthestorageengine.Thestorageengineisresponsibleforpersistingthedocumentsondisk,holdingcopiesinmemory,providingindexesandcachestospeedupqueries.
Uptoversion3.1ArangoDBonlysupportedmemorymappedfiles(MMFiles)assolestorageengine.Beginningwith3.2ArangoDBhassupportforpluggablestorageengines.ThesecondsupportedengineisRocksDBfromFacebook.
MMFiles RocksDB
default optional
datasetneedstofitintomemory workwithasmuchdataasfitsondisk
indexesinmemory hotsetinmemory,dataandindexesondisk
slowrestartduetoindexrebuilding faststartup(norebuildingofindexes)
volatilecollections(onlyinmemory,optional) collectiondataalwayspersisted
collectionlevellocking(writesblockreads) concurrentreadsandwrites
Blogarticle:ComparingnewRocksDBandMMFilesstorageengines
RocksDBisanembeddablepersistentkey-valuestore.Itisalogstructuredatabaseandisoptimizedforfaststorage.
TheMMFilesengineisoptimizedfortheuse-casewherethedatafitsintothemainmemory.Itallowsforveryfastconcurrentreads.However,writesblockreadsandlockingisoncollectionlevel.Indexesarealwaysinmemoryandarerebuiltonstartup.Thisgivesbetterperformancebutimposesalongerstartuptime.
TheRocksDBengineisoptimizedforlargedata-setsandallowsforasteadyinsertperformanceevenifthedata-setismuchlargerthanthemainmemory.Indexesarealwaysstoredondiskbutcachesareusedtospeedupperformance.RocksDBusesdocument-levellocksallowingforconcurrentwrites.Writesdonotblockreads.Readsdonotblockwrites.
Theenginemustbeselectedforthewholeserver/cluster.Itisnotpossibletomixengines.Thetransactionhandlingandwrite-ahead-logformatintheindividualenginesisverydifferentandthereforecannotbemixed.
RocksDB
Advantages
RocksDBisaveryflexibleenginethatcanbeconfiguredforvarioususecases.
ThemainadvantagesofRocksDBare
document-levellockssupportforlargedata-setspersistentindexes
Caveats
RocksDBallowsconcurrentwrites.However,whentouchingthesamedocumentawriteconflictisraised.ThiscannothappenwiththeMMFilesengine,thereforeapplicationsthatswitchtoRocksDBneedtobepreparedthatsuchexceptioncanarise.ItispossibletoexclusivelylockcollectionswhenexecutingAQL.Thiswillavoidwriteconflictsbutalsoinhibitsconcurrentwrites.
Currently,anotherrestrictionisduetothetransactionhandlinginRocksDB.Transactionsarelimitedintotalsize.Ifyouhaveastatementmodifyingalotofdocumentsitisnecessarytocommitdatainbetween.ThiswillbedoneautomaticallyforAQLbydefault.
Performance
RocksDBisabasedonlog-structuredmergetree.Agoodintroductioncanbefoundin:
StorageEngines
482
http://www.benstopford.com/2015/02/14/log-structured-merge-trees/https://blog.acolyer.org/2014/11/26/the-log-structured-merge-tree-lsm-tree/
Thebasicideaisthatdataisorganizedinlevelswereeachlevelisafactorlargerthantheprevious.Newdatawillresideinsmallerlevelswhileolddataismoveddowntothelargerlevels.Thisallowstosupporthighrateofinsertsoveranextendedperiod.Inprincipleitispossiblethatthedifferentlevelsresideondifferentstoragemedia.ThesmalleronesonfastSSD,thelargeronesonbiggerspinningdisks.
RocksDBitselfprovidesalotofdifferentknobstofinetunethestorageengineaccordingtoyouruse-case.ArangoDBsupportsthemostcommononesusingtheoptionsbelow.
Performancereportsforthestorageenginecanbefoundhere:
https://github.com/facebook/rocksdb/wiki/performance-benchmarkshttps://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide
ArangoDBoptions
ArangoDBhasacacheforthepersistentindexesinRocksDB.Thetotalsizeofthiscacheiscontrolledbytheoption
--cache.size
RocksDBalsohasacachefortheblocksstoredondisk.Thesizeofthiscacheiscontrolledbytheoption
--rocksdb.block-cache-size
ArangoDBdistributestheavailablememoryequallybetweenthetwocachesbydefault.
ArangoDBchoosesasizeforthevariouslevelsinRocksDBthatissuitableforgeneralpurposeapplications.
RocksDBlogstrutureddatalevelshaveincreasingsize
MEM:--
L0:--
L1:----
L2:--------
...
NeworupdatedDocumentsarefirststoredinmemory.Ifthismemtablereachesthelimitgivenby
--rocksdb.write-buffer-size
itwillconvertedtoanSSTfileandinsertedatlevel0.
Thefollowingoptioncontrolsthesizeofeachlevelandthedepth.
--rocksdb.num-levelsN
LimitsthenumberoflevelstoN.Bydefaultitis7andthereisseldomareasontochangethis.Anewlevelisonlyopenedifthereistoomuchdatainthepreviousone.
--rocksdb.max-bytes-for-level-baseB
L0willholdatmostBbytes.
--rocksdb.max-bytes-for-level-multiplierM
EachlevelisatmostMtimesasmuchbytesasthepreviousone.ThereforethemaximumnumberofbytesforlevelLcanbecalculatedas
max-bytes-for-level-base*(max-bytes-for-level-multiplier^(L-1))
StorageEngines
483
Future
RocksDBimposesalimitonthetransactionsize.Itisoptimizedtohandlesmalltransactionsveryefficiently,butiseffectivelylimitingthetotalsizeoftransactions.
ArangoDBcurrentlyusesRocksDB'stransactionstoimplementtheArangoDBtransactionhandling.ThereforethesamerestrictionsapplyforArangoDBtransactionswhenusingtheRocksDBengine.
WewillimprovethisbyintroducingdistributedtransactionsinafutureversionofArangoDB.ThiswillallowhandlinglargetransactionsasaseriesofsmallRocksDBtransactionsandhenceremovingthesizerestriction.
StorageEngines
484
ReleaseNotes
WhatsNew
WhatsNewin3.3WhatsNewin3.2WhatsNewin3.1WhatsNewin3.0WhatsNewin2.8WhatsNewin2.7WhatsNewin2.6WhatsNewin2.5WhatsNewin2.4WhatsNewin2.3WhatsNewin2.2WhatsNewin2.1
KnownIssues
KnownIssuesin3.2
Incompatiblechanges
AlsoseeUpgradingintheAdministrationchapter.
Incompatiblechangesin3.3Incompatiblechangesin3.2Incompatiblechangesin3.1Incompatiblechangesin3.0Incompatiblechangesin2.8Incompatiblechangesin2.7Incompatiblechangesin2.6Incompatiblechangesin2.5Incompatiblechangesin2.4Incompatiblechangesin2.3
Releasenotes
485
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB3.3.ArangoDB3.3alsocontainsseveralbugfixesthatarenotlistedhere.
Datacenter-to-datacenterreplication(DC2DC)
Everycompanyneedsadisasterrecoveryplanforallimportantsystems.Thisistruefromsmallunitslikesingleprocessesrunninginsomecontainertothelargestdistributedarchitectures.Fordatabasesinparticularthisusuallyinvolvesamixtureoffault-tolerance,redundancy,regularbackupsandemergencyplans.Thelargeradatastore,themoredifficultitistocomeupwithagoodstrategy.
Therefore,itisdesirabletobeabletorunadistributeddatabaseinonedata-centerandreplicatealltransactionstoanotherdata-centerinsomeway.Often,transactionlogsareshippedoverthenetworktoreplicateeverythinginanother,identicalsystemintheotherdata-center.Somedistributeddatastoreshavebuilt-insupportformultipledata-centerawarenessandcanreplicatebetweendata-centersinafullyautomaticfashion.
ArangoDB3.3takesanevolutionarystepforwardbyintroducingmulti-data-centersupport,whichisasynchronousdata-centertodata-centerreplication.Oursolutionisasynchronousandscalestoarbitraryclustersizes,providedyournetworklinkbetweenthedata-centershasenoughbandwidth.Itisfault-tolerantwithoutasinglepointoffailureandincludesalotofmetricsformonitoringinaproductionscenario.
DC2DCisavailableintheEnterpriseedition.
Encryptedbackups
ArangodumpcannowcreateencryptedbackupsusingAES256forencryption.Theencryptionkeycanbereadfromafileorfromageneratorprogram.Itworksinsingleserverandclustermode.
Examplefornon-encryptedbackup(everyonewithaccesstothebackupwillbeabletoreadit):
arangodump--collection"secret"dump
Inordertocreateanencryptedbackup,addthe--encryption.keyfileoptionwheninvokingarangodump:
arangodump--collection"secret"dump--encryption.keyfile~/SECRET-KEY
Thekeymustbeexactly32byteslong(requiredbytheAESblockcipher).
Notethatarangodumpwillnotstorethekeyanywhere.Itistheresponsibilityoftheusertofindasafeplaceforthekey.However,arangodumpwillstoretheusedencryptionmethodinafilenamedENCRYPTIONinthedumpdirectory.Thatwayarangorestorecanlaterfindoutwhetheritisdealingwithanencrypteddumpornot.
Tryingtorestoretheencrypteddumpwithoutspecifyingthekeywillfail:
arangorestore--collection"secret-collection"dump--create-collectiontrue
arangorestorewillcomplainwith:
thedumpdataseemstobeencryptedwithaes-256-ctr,butnokeyinformationwasspecifiedtodecryptthedumpitisrecommendedtospecifyeither--encryption.key-fileor--encryption.key-generatorwheninvokingarangorestorewithanencrypteddump
Itisrequiredtousetheexactsamekeywhenrestoringthedata.Againthisisdonebyprovidingthe--encryption.keyfileparameter:
arangorestore--collection"secret-collection"dump--create-collectiontrue--encryption.keyfile~/SECRET-KEY
WhatsNewin3.3
486
Usingadifferentkeywillleadtothebackupbeingnon-recoverable.
NotethatencryptedbackupscanbeusedtogetherwiththealreadyexistingRocksDBencryption-at-restfeature,buttheycanalsobeusedfortheMMFilesengine,whichdoesnothaveencryption-at-rest.
EncryptedbackupsareavailableintheEnterpriseedition.
Server-levelreplicationArangoDBsupportsasynchronousreplicationfunctionalitysinceversion1.4,butreplicatingfromamasterserverwithmultipledatabasesrequiredmanualsetupontheslaveforeachindividualdatabasetoreplicate.Whenanewdatabasewascreatedonthemaster,oneneededtotakeactionontheslavetoensurethatdataforthatdatabasegotactuallyreplicated.Replicationontheslavealsowasnotawareofwhenadatabasewasdroppedonthemaster.
3.3addsserver-levelreplication,whichwillreplicatethecurrentandfuturedatabasesfromthemastertotheslaveautomaticallyaftertheinitialsetup.
Inordertosetupglobalreplicationona3.3slaveforalldatabasesofagiven3.3master,thereisnowtheso-calledglobalApplier.Ithasthesameinterfaceastheexistingapplier,butitwillreplicatefromalldatabasesofthemasterandnotjustasingleone.
Inordertostartthereplicationontheslaveandmakeitreplicatealldatabasesfromagivenmaster,usethesecommandsontheslave:
varreplication=require("@arangodb/replication");
replication.setupReplicationGlobal({
endpoint:"tcp://127.0.0.1:8529",
username:"root",
password:"",
autoStart:true
});
Tocheckiftheapplierisrunning,alsousetheglobalApplierobject:
replication.globalApplier.state().state
Theserver-levelreplicationrequiresboththemasterandslaveserverstoArangoDBversion3.3orhigher.
Asynchronousfailover
Aresilientsetupcannoweasilybeachievedbyrunningapairofconnectedservers,ofwhichoneinstancebecomesthemasterandtheotheranasynchronouslyreplicatingslave,withautomaticfailoverbetweenthem.
Twoserversareconnectedviaasynchronousreplication.Oneoftheserversiselectedleader,andtheotheroneismadeafollowerautomatically.Atstartup,thetwoserversfightforleadership.Thefollowerwillautomaticallystartreplicationfromthemasterforallavailabledatabases,usingtheserver-levelreplicationintroducedin3.3.
Whenthemastergoesdown,thisisautomaticallydetectedbyanagencyinstance,whichisalsostartedinthismode.Thisinstancewillmakethepreviousfollowerstopitsreplicationandmakeitthenewleader.
Thefollowerwillautomaticallydenyallreadandwriterequestsfromclientapplications.Onlythereplicationitselfisallowedtoaccessthefollower'sdatauntilthefollowerbecomesanewleader.
Whensendingarequesttoreadorwritedataonafollower,thefollowerwillalwaysrespondwithHTTP503(Serviceunavailable)andprovidetheaddressofthecurrentleader.Clientapplicationsanddriverscanusethisinformationtothenmakeafollow-uprequesttotheproperleader:
HTTP/1.1503ServiceUnavailable
X-Arango-Endpoint:http://[::1]:8531
....
WhatsNewin3.3
487
Clientapplicationscanalsodetectwhothecurrentleaderandthefollowersarebycallingthe/_api/cluster/endpointsRESTAPI.ThisAPIisaccessibleonleadersandfollowersalike.
TheArangoDBstartersupportsstartingtwoserverswithasynchronousreplicationandfailoveroutofthebox.
ThearangojsdriverforJavaScript,theGodriverandtheJavadriverforArangoDBsupportautomaticfailoverincasethecurrentlyaccessedserverendpointrespondswithHTTP503.
Blogarticles:
UsingAutomaticfailoverforresilientsingleserversIntroducingthenewArangoDBJavadriverwithloadbalancingandadvancedfallback
RocksDBthrottlingArangoDB3.3allowswriteoperationstotheRocksDBenginebethrottled,inordertopreventlongerwritestalls.Thethrottlingisadaptive,meaningthatitautomaticallyadaptstotheactualwriterate.Thisresultsinmuchmorestableresponsetimes,whichisbetterforclientapplicationsandclusterhealthtests,becausetimeoutscausedbywritestallsarelesslikelytooccurandtheserverthusnotmistakenlyassumedtobedown.
Blogarticle:RocksDBsmoothingforArangoDBcustomers
Fastershardcreationincluster
Whenusingacluster,onenormallywantsresilience,soreplicationFactorissettoatleast2.Thenumberofshardsisoftensettoratherhighvalueswhencreatingcollections.
Creatingacollectionintheclusterwillmakethecoordinatorstorethesetupmetadataofthenewcollectionintheagencyfirst.Subsequentiallyalldatabaseserversoftheclusterwilldetectthatthereisworktodoandwillbegincreatingtheshards.Thiswillfirsthappenfortheshardleaders.Foreachshardleaderthatfinisheswiththesetup,thesynchronousreplicationwithitsfollowersisthenestablished.Thatwillmakesurethateveryfuturedatamodificationwillnotbecomeeffectiveontheleaderonly,butalsoonallthefollowers.
In3.3thissetupprotocolhasgotsomeshortcutsfortheinitialshardcreation,whichspeedsupcollectioncreationbyroughly50to60percent.
LDAPauthenticationTheLDAPauthenticationmoduleintheEnterpriseeditionhasbeenenhanced.Thefollowingoptionshavebeenaddedtoit:
theoption--server.local-authenticationcontrolswhetherthelocal_userscollectionisalsousedforlookingupusers.Thisisalsothedefaultbehavior.IftheauthenticationshallberestrictedtojusttheLDAPdirectory,theoptioncanbesettotrue,andarangodwillthennotmakeanyqueriestoits_userscollectionwhenlookingupusers.
theoption--server.authentication-timeoutcontrolstheexpirationtimeforcachedLDAPuserinformationentriesinarangod.
basicrolesupporthasbeenaddedfortheLDAPmoduleintheEnterpriseedition.NewconfigurationoptionsforLDAPin3.3are:
--ldap.roles-attribute-name
--ldap.roles-transformation
--ldap.roles-search
--ldap.roles-include
--ldap.roles-exclude
--ldap.superuser-role
PleaserefertoLDAPforadetailedexplanation.
Miscellaneousfeatures
WhatsNewin3.3
488
whencreatingacollectioninthecluster,thereisnowanoptionalparameterenforceReplicationFactor:whenset,thisparameterenforcesthatthecollectionwillonlybecreatediftherearenotenoughdatabaseserversavailableforthedesiredreplicationFactor.
AQLDISTINCTisnotchangingtheorderofprevious(sorted)results
PreviouslytheimplementationofAQLdistinctstoredallencounteredvaluesinahashtableinternally.Whendone,thefinalresultswerereturnedintheorderdictatedbythehashtablethatwasusedtostorethekeys.Thisorderwasmoreorlessunpredictable.Thoughthiswasdocumentedbehavior,itwasinconvenientforendusers.
3.3nowdoesnotchangethesortorderoftheresultanymorewhenDISTINCTisused.
SeveralAQLfunctionshavebeenimplementedinC++,whichcanhelpsavememoryandCPUtimeforconvertingthefunctionargumentsandresults.Thefollowingfunctionshavebeenported:
LEFTRIGHTSUBSTRINGTRIMMATCHES
TheArangoShellpromptsubstitutioncharactershavebeenextended.Nowthefollowingextrasubstitutionscanbeusedforthearangoshprompt:
'%t':currenttimeastimestamp'%a':elpasedtimesinceArangoShellstartinseconds'%p':durationoflastcommandinseconds
Forexample,toshowtheexecutiontimeofthelastcommandexecutedinarangoshintheshell'sprompt,startarangoshusing:
arangosh--console.prompt"%E@%d%p>"
Therearenewstartupoptionsfortheloggingtoaiddebugginganderrorreporting:
--log.role:willshowone-lettercodeofserverrole(A=agent,C=coordinator,...)Thisisespeciallyusefulwhenaggregatinglogs.
Theexistingrolesusedinlogsare:
U:undefined/unclear(usedatstartup)S:singleserverC:coordinatorP:primaryA:agent
--log.line-numbertrue:thisoptionwillnowadditionallyshowthenameoftheC++functionthattriggeredthelogmessage(filenameandlinenumberwerealreadyloggedinpreviousversions)
--log.thread-nametrue:thisnewoptionwilllogthenameoftheArangoDBthreadthattriggeredthelogmessage.WillhavemeaningfuloutputonLinuxonly
maketheArangoShell(arangosh)refillitscollectioncachewhenayet-unknowncollectionisfirstaccessed.Thisfixesthefollowingproblemwhenworkingwiththeshellwhileinanothershellorbyanotherprocessanewcollectionisadded:
arangosh1>db._collections();//shell1listsallcollections
arangosh2>db._create("test");//shell2nowcreatesanewcollection'test'
arangosh1>db.test.insert({});//shell1isnotawareofthecollectioncreated
//inshell2,sotheinsertwillfail
WhatsNewin3.3
489
IncompatiblechangesinArangoDB3.3ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB3.3,andadjustanyclientprogramsifnecessary.
ThefollowingincompatiblechangeshavebeenmadeinArangoDB3.3:
AQL:duringatraversalifavertexisnotfound,arangodwillnotloganerrorandcontinuewithaNULLvalue,butwillinsteadregisterawarningatthequeryandcontinuewithaNULLvalue.
Ifanon-existingvertexisreferencedfromatraversal,itisnotdesirabletologerrorsasArangoDBcanstoreedgespointingtonon-existingvertices(whichisperfectlyvalidifthelow-levelinsertAPIsareused).Aslinkingtonon-existingverticesmayindicateanissuein/withthedatamodelortheclientapplication,thewarningisregisteredinthequerysoclientapplicationshaveaccesstoit.
ArangoDBusernamesmustnotstartwiththestring:role:.
Thestartupconfigurationparameter--cluster.my-iddoesnothaveanyeffectin3.3.Forcompatibilityreasons,ArangoDB3.3willnotfailonstartupiftheoptionisstillusedintheconfiguration,butitwillsilentlyignorethisoption.
Thestartupconfigurationparameter--cluster.my-local-infoisdeprecatednow.Usingitwillmakearangodlogawarningonstartup.
Serverstartup:therecommendedvaluefortheLinuxkernelsettingin/proc/sys/vm/max_map_countwasincreasedtoavalueeighttimesashighasin3.2.arangodcomparesatstartupiftheeffectivevalueofthissettingispresumablytoolow,anditwillissueawarninginthiscase,recommendingtoincreasethevalue.
Thisisnowmorelikelytohappenthaninpreviousversions,astherecommendedvalueisnoweighttimeshigherthanin3.2.Thestartupwarningswilllooklikethis(withactualnumbersvarying):
WARNING{memory}maximumnumberofmemorymappingsperprocessis65530,whichseemstoolow.itisrecommendedtosetit
toatleast512000
PleaserefertotheLinuxkerneldocumentationformoreinformationonthissetting.ThischangeonlyaffectstheLinuxversionofArangoDB.
Clienttools
Theoption--recycle-idshasbeenremovedfromthearangorestorecommand.Usingthisoptioncouldhaveledtoproblemsontherestore,withpotentialidconflictsbetweentheoriginatingserver(thesourcedumpserver)andthetargetserver(therestoreserver).
Theoption--compatparameterhasbeenremovedfromthearangodumpcommandandthe/_api/replication/dumpRESTAPIendpoint.InordertocreateadumpfromanArangoDB2.8instance,pleaseuseanolderversionoftheclienttools.OlderArangoDBversionsarenolongerbesupportedbythearangodumpandarangorestorebinariesshippedwith3.3.
Miscellaneous
TheminimumsupportedcompilerforcompilingArangoDBfromsourceisnowg++5.4(bumpedupfromg++4.9).ThischangeonlyaffectsusersthatcompileArangoDBontheirown.
Incompatiblechangesin3.3
490
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB3.2.ArangoDB3.2alsocontainsseveralbugfixesthatarenotlistedhere.
Storageengines
ArangoDB3.2offerstwostorageengines:
thealways-existingmemory-mappedfilesstorageengineanewstorageenginebasedonRocksDB
Memory-mappedfilesstorageengine(MMFiles)
Theformerstorageengine(namedMMFilesenginehenceforth)persistsdatainmemory-mappedfiles.
Anydatachangesaredonefirstintheengine'swrite-aheadlog(WAL).TheWALisreplayedafteracrashsotheengineoffersdurabilityandcrash-safety.DatafromtheWALiseventuallymovedtocollection-specificdatafiles.Thefilesarealwayswritteninanappend-onlyfashion,sodatainfilesisneveroverwritten.Obsoletedatainfileswilleventuallybepurgedbybackgroundcompactionthreads.
Mostofthisengine'sindexesarebuiltinRAM.Whenacollectionisloaded,thisrequiresrebuildingtheindexesinRAMfromthedatastoredondisk.TheMMFilesenginehascollection-levellocking.
Thisstorageengineisagoodchoicewhendata(includingtheindexes)canfitintheserver'savailableRAM.Ifthesizeofdataplusthein-memoryindexesexceedsthesizeoftheavailableRAM,thenthisenginemaytrytoallocatemorememorythanavailable.Thiswilleithermaketheoperatingsystemswapoutpartsofthedata(andcausediskI/O)or,whennoswapspaceisconfigured,invoketheoperatingsystem'sout-of-memoryprocesskiller.
Thelockingstrategyallowsparallelreadsandisoftengoodenoughinread-mostlyworkloads.Writesneedexclusivelocksonthecollections,sotheycanblockotheroperationsinthesamecollection.Thelockingstrategyalsoprovidestransactionalconsistencyandisolation.
RocksDBstorageengine
TheRocksDBstorageengineisnewinArangoDB3.2.Itisdesignedtostoredatasetsthatarebiggerthantheserver'savailableRAM.Itpersistsalldata(includingtheindexes)inaRocksDBinstance.
ThatmeansanydocumentreadorwriteoperationswillbeansweredbyRocksDBunderthehood.RocksDBwillservethedatafromitsownin-RAMcachesorfromdisk.TheRocksDBenginehasawrite-aheadlog(WAL)andusesbackgroundthreadsforcompaction.Itsupportsdatacompression.
TheRocksDBstorageenginehasdocument-levellocking.Readoperationsdonotblockandareneverblockedbyotheroperations.Writeoperationsonlyblockwritesonthesamedocuments/indexvalues.Becausemultiplewriterscanoperateinparallelonthesamecollection,thereisthepossibilityofwrite-writeconflicts.Ifsuchwriteconflictisdetected,oneofthewriteoperationsisabortedwitherror1200("conflict").Clientapplicationscantheneitheraborttheoperationorretry,basedontherequiredconsistencysemantics.
Storageengineselection
ThestorageenginetouseinanArangoDBclusterorasingle-serverinstancemustbeselectedinitially.ThedefaultstorageengineinArangoDB3.2istheMMFilesengineifnostorageengineisselectedexplicitly.Thisensuresallusersupgradingfromearlierversionscancontinuewiththewell-knownMMFilesengine.
Toselectthestorage-engine,thereistheconfigurationoption--server.storage-engine.Itcanbesettoeithermmfiles,rocksdborauto.Whilethefirsttwovalueswillexplicitlyselectastorageengine,theautooptionwillautomaticallychoosethestorageenginebasedonwhichstorageenginewaspreviouslyselected.Ifnoenginewasselectedpreviously,autowillselecttheMMFilesengine.Ifanenginewaspreviouslyselected,theselectionwillbewrittentoafileENGINEintheserver'sdatabasedirectoryandwillbereadfromthereatanysubsequentserverstarts.
WhatsNewin3.2
491
Oncethestorageenginewasselected,theselectioncannotbechangedbyadjusting--server.storage-engine.Inordertoswitchtoanotherstorageengine,itisrequiredtore-starttheserverwithanother(empty)databasedirectory.Inordertousedatacreatedwiththeotherstorageengine,itisrequiredtodumpthedatafirstwiththeoldengineandrestoreitusingthenewstorageengine.Thiscanbeachievedviainvokingarangodumpandarangorestore.
UnlikeinMySQL,thestorageengineselectioninArangoDBisforanentireclusteroranentiresingle-serverinstance.Alldatabasesandcollectionswillusethesamestorageengine.
RocksDBstorageengine:supportedindextypes
TheexistingindexesintheRocksDBengineareallpersistent.Thefollowingindexesaresupportedthere:
primary:thistypeofindexisautomaticallycreated.Itindexes_id/_key
edge:thisindexisautomaticallycreatedforedgecollections.Itindexes_fromand_to
hash,skiplist,persistent:theseareuser-definedindexes,Despitetheirnames,theyareneitherhashnorskiplistindexes.TheseindextypesmaptothesameRocksDB-basedsortedindeximplementation.Thesameistrueforthe"persistent"index.Thenames"hash","skiplist"and"persistent"areonlyusedforcompatibilitywiththeMMFilesenginewheretheseindexesexistedinpreviousandthecurrentversionofArangoDB.
geo:user-definedindexforproximitysearches
fulltext:user-definedsortedrevertedindexonwordsoccurringindocuments
SatelliteCollections
WithSatelliteCollections,youcandefinecollectionstoshardtoaclusterandcollectionstoreplicatetoeachmachine.TheArangoDBqueryoptimizerknowswhereeachshardislocatedandsendstherequeststotheDBServersinvolved,whichthenexecutesthequery,locally.Withthisapproach,networkhopsduringjoinoperationsonshardedcollectionscanbeavoidedandresponsetimescanbeclosetothatofasingleinstance.
SatellitecollectionsareavailableintheEnterpriseedition.
MemorymanagementmakearangodstartwithlessV8JavaScriptcontexts
Thisspeedsuptheserverstartandmakesarangoduselessmemoryatstart.WheneveraV8contextisneededbyaFoxxactionorsomeotherJavaScriptoperationandthereisnousableV8context,anewcontextwillbecreateddynamicallynow.
Upto--javascript.v8-contextsV8contextswillbecreated,sothisoptionwillchangeitsmeaning.PreviouslyasmanyV8contextsasspecifiedbythisoptionwerecreatedatserverstart,andthenumberofV8contextsdidnotchangeatruntime.NowuptothisnumberofV8contextswillbeinuseatthesametime,buttheactualnumberofV8contextsisdynamic.
ThegarbagecollectorthreadwillautomaticallydeleteunusedV8contextsafterawhile.Thenumberofsparecontextswillgodowntoasfewasconfiguredinthenewoption--javascript.v8-contexts-minimum.ActuallythatmanyV8contextsarealsocreatedatserverstart.
ThefirstfewrequestsinnewV8contextsmaytakelongerthanincontextsthathavebeentherealready.PerformancemaythereforesufferabitfortheinitialrequestssenttoArangoDBorwhenthereareonlyfewbutperformance-criticalsituationsinwhichnewV8contextsneedtobecreated.Ifthisisaconcern,itcaneasilybefixedbysetting--javascipt.v8-contexts-minimumand--javascript.v8-contextstoarelativelyhighvalue,whichwillguaranteethatmanynumberofV8contextstobecreatedatstartupandkeptaroundevenwhenunused.
WaitingforanunusedV8contextwillnowalsoabortandwritealogmessageincasenoV8contextcanbeacquired/createdafter60seconds.
thenumberofpendingoperationsinarangodcannowbelimitedtoaconfigurablenumber.Ifthisnumberisexceeded,theserverwillnowrespondwithHTTP503(serviceunavailable).Themaximumsizeofpendingoperationsiscontrolledviathestartupoption--server.maximal-queue-size.Settingitto0means"nolimit".
WhatsNewin3.2
492
thein-memorydocumentrevisionscachewasremovedentirelybecauseitdidnotprovidetheexpectedbenefits.The3.1implementationshadoweddocumentdatainRAM,whichincreasedtheserver'sRAMusagebutdidnotspeedupdocumentlookupstoomuch.
Thisalsoobsoletesthestartupoptions--database.revision-cache-chunk-sizeand--database.revision-cache-target-size.
TheMMFilesenginenowdoesnotuseadocumentrevisionscachebuthasin-memoryindexesandmapsdocumentstoRAMautomaticallyviammapwhendocumentsareaccessed.TheRocksDBenginehasitsownmechanismforcachingaccesseddocuments.
CommunicationLayerHTTPresponsesreturnedbyarangodwillnowincludetheextraHTTPheaderx-content-type-options:nosnifftoworkaroundacross-sitescriptingbuginMSIE
thedefaultvaluefor--ssl.protocolwaschangedfromTLSv1toTLSv1.2.Whennotexplicitlyset,arangodandallclienttoolswillnowuseTLSv1.2.
theJSONdatainallincomingHTTPrequestsinnowvalidatedforduplicateattributenames.
IncomingJSONdatawithduplicateattributenameswillnowberejectedasinvalid.PreviousversionsofArangoDBonlyvalidatedtheuniquenessofattributenamesinsideincomingJSONforsomeAPIendpoints,butnotconsistentlyforallAPIs.
InternalJavaScriptRESTactionswillnowhidetheirstacktracestotheclientunlessinHTTPresponses.Insteadtheywillalwayslogtothelogfile.
JavaScript
updatedV8versionto5.7.0.0
changeundocumentedbehaviourincaseofinvalidrevisionidsinIf-MatchandIf-None-Matchheadersfrom400(BAD)to412(PRECONDITIONFAILED).
changedefaultstringtruncationlengthfrom80charactersto256charactersforprint/printShellfunctionsinArangoShellandarangod.Thiswillemitlongerprefixesofstringvaluesbeforetruncatingthemwith...,whichishelpfulfordebugging.ThischangeismostlyusefulwhenusingtheArangoShell(arangosh).
the@arangodbmodulenowprovidesatimefunctionwhichreturnsthecurrenttimeinsecondsasafloatingpointvaluewithmicrosecondprecision.
FoxxThereisnowanofficialHTTPAPIformanagingservices,allowingservicestobeinstalled,modified,uninstalledandreconfiguredwithouttheadministrativewebinterface.
ItisnowpossibletouploadasingleJavaScriptfileinsteadofaziparchiveifyourservicerequiresnoconfiguration,additionalfilesorsetup.Aminimalmanifestwillbegeneratedautomaticallyuponinstallationandtheuploadedfilewillbeusedastheservice'smainentrypoint.
DistributedGraphProcessing
WeaddedsupportforexecutingdistributedgraphalgorithmsakaPregel.Userscanrunarbitraryalgorithmsonanentiregraph,includinginclustermode.Weimplementedanumberofalgorithmsforvariouswell-knowngraphmeasures:
ConnectedComponentsPageRankShortestPathsCentralityMeasures(CentralityandBetweeness)
WhatsNewin3.2
493
CommunityDetection(viaLabelPropagation,Speakers-ListenersLabelPropagationorDMID)Userscancontributetheirownalgorithms
AQL
Optimizerimprovements
GeoindexesarenowimplicitlyandautomaticallyusedwhenusingappropriateSORT/FILTERstatementsinAQL,withouttheneedtousethesomewhatlimitedspecial-purposegeoAQLfunctionsNEARorWITHIN.
ComparedtousingthespecialpurposeAQLfunctionsthisapproachhastheadvantagethatitismorecomposable,andwillalsohonoranyLIMITvaluesusedintheAQLquery.
ThespecialpurposeNEARAQLfunctioncannowbesubstitutedwiththefollowingAQL(providedthereisageoindexpresentonthedoc.latitudeanddoc.longitudeattributes):
FORdocingeoSort
SORTDISTANCE(doc.latitude,doc.longitude,0,0)
LIMIT5
RETURNdoc
WITHINcanbesubstitutedwiththefollowingAQL:
FORdocingeoFilter
FILTERDISTANCE(doc.latitude,doc.longitude,0,0)<2000
RETURNdoc
Miscellaneousimprovements
addedREGEX_REPLACEAQLfunction
REGEX_REPLACE(text,search,replacement,caseInsensitive)→string
Replacethepatternsearchwiththestringreplacementinthestringtext,usingregularexpressionmatching.
text(string):thestringtosearchinsearch(string):aregularexpressionsearchpatternreplacement(string):thestringtoreplacethesearchpatternwithreturnsstring(string):thestringtextwiththesearchregexpatternreplacedwiththereplacementstringwhereverthepatternexistsintext
addednewstartupoption--query.fail-on-warningtomakeAQLqueriesabortinsteadofcontinuingwithwarnings.
Whensettotrue,thiswillmakeanAQLquerythrowanexceptionandabortincaseawarningoccurs.Thisoptionshouldbeusedindevelopmenttocatcherrorsearly.Ifsettofalse,warningswillnotbepropagatedtoexceptionsandwillbereturnedwiththequeryresults.Thestartupoptioncanalsobeoverridenonaperquery-level.
theslowquerylistnowcontainsthevaluesofbindvariablesusedintheslowqueries.Bindvariablesarealsoprovidedforthecurrentlyrunningqueries.Thishelpsdebuggingsloworblockingqueriesthatusedynamiccollectionnamesviabindparameters.
AQLbreakingchangeincluster:TheSHORTEST_PATHstatementusingedgecollectionnamesinsteadofagraphnamesnowrequirestoexplicitlynamethevertexcollectionnameswithintheAQLqueryinthecluster.ItcanbedonebyaddingWITH<name>atthebeginningofthequery.
Example:
FORv,eINOUTBOUNDSHORTEST_PATH@startTO@targetedges[...]
Nowhastobe:
WITHvertices
FORv,eINOUTBOUNDSHORTEST_PATH@startTO@targetedges[...]
WhatsNewin3.2
494
Thischangeisduetoavoiddeadlocksitationsinclusteredcase.Anerrorstatingtheaboveisincluded.
Clienttools
addeddataexporttool,arangoexport.
arangoexportcanbeusedtoexportcollectionstojson,jsonlorxmlandexportagraphorcollectionstoxgmml.
added"jsonl"asinputfiletypeforarangoimp
added--translateoptionforarangoimptotranslateattributenamesfromtheinputfilestoattriubtenamesexpectedbyArangoDB
The--translateoptioncanbespecifiedmultipletimes(oncepertranslationtobeexecuted).Thefollowingexamplerenamesthe"id"columnfromtheinputfileto"_key",andthe"from"columnto"_from",andthe"to"columnto"_to":
arangoimp--typecsv--filedata.csv--translate"id=_key"--translate"from=_from"--translate"to=_to"
--translateworksforCSVandTSVinputsonly.
added--threadsoptiontoarangoimptospecifythenumberofparallelimportthreads
changeddefaultvalueforclienttoolsoption--server.max-packet-sizefrom128MBto256MB.thisallowstransferringbiggerresultsetsfromtheserverwithouttheclienttoolsrejectingthemasinvalid.
Authentication
addedLDAPauthentication(Enterpriseonly)
Authorization
addedreadonlymodeforuserscollectionlevelauthorizationrights
Readmoreintheoverview.
Foxxthecookiesessiontransportnowsupportsalloptionssupportedbythecookiemethodoftheresponseobject.
it'snowpossibletoprovideyourownversionofthegraphql-syncmodulewhenusingtheGraphQLextensionsforFoxxbypassingacopyofthemoduleusingthenewgraphqloption.
customAPIendpointscannowbetaggedusingthetagmethodtogenerateacleanerSwaggerdocumentation.
MiscellaneousChanges
arangodnowvalidatesseveralOS/environmentsettingsonstartupandwarnsifthesettingsarenon-ideal.Itadditionallywillprintoutwaystoremedytheoptions.
MostofthechecksareexecutedonLinuxsystemsonly.
added"deduplicate"attributeforarrayindexes,whichcontrolswhetherinsertingduplicateindexvaluesfromthesamedocumentintoauniquearrayindexwillleadtoanerrorornot:
//withdeduplicate=true,whichisthedefaultvalue:
db._create("test");
db.test.ensureIndex({type:"hash",fields:["tags[*]"],deduplicate:true});
db.test.insert({tags:["a","b"]});
WhatsNewin3.2
495
db.test.insert({tags:["c","d","c"]});//willwork,becausededuplicate=true
db.test.insert({tags:["a"]});//willfail
//withdeduplicate=false
db._create("test");
db.test.ensureIndex({type:"hash",fields:["tags[*]"],deduplicate:false});
db.test.insert({tags:["a","b"]});
db.test.insert({tags:["c","d","c"]});//willnotwork,becausededuplicate=false
db.test.insert({tags:["a"]});//willfail
WhatsNewin3.2
496
KnownIssuesThefollowingknownissuesarepresentinthisversionofArangoDBandwillbefixedinfollow-upreleases:
RocksDBstorageengine
TheRocksDBstorageengineisintentionallymissingthefollowingfeaturesthatarepresentintheMMFilesengine:
thedatafiledebugger(arango-dfdb)cannotbeusedwiththisstorageengine
RocksDBhasitsowncrashrecoverysousingthedfdbwillnotmakeanysensehere.
APIsthatreturncollectionpropertiesorfigureswillreturnslightlydifferentattributesfortheRocksDBenginethanfortheMMFilesengine.Forexample,theattributesjournalSize,doCompact,indexBucketsandisVolatilearepresentintheMMFilesenginebutnotintheRocksDBengine.ThememoryusagefiguresreportedforcollectionsintheRocksDBengineareestimatevalues,whereastheyareexactfortheMMFilesengine.
theRocksDBenginedoesnotsupportsomeoperationswhichonlymakesenseinthecontextoftheMMFilesengine.Theseare:
therotatemethodoncollectionstheflushmethodforWALfiles
theRocksDBstorageenginedoesnotsupportvolatilecollections
transactionsarelimitedinsize.Transactionsthatgettoobig(intermsofnumberofoperationsinvolvedorthetotalsizeofdatamodifiedbythetransaction)willbecommittedautomatically.EffectivelythismeansthatbigusertransactionsaresplitintomultiplesmallerRocksDBtransactionsthatarecommittedindividually.TheentireusertransactionwillnotnecessarilyhaveACIDpropertiesinthiscase.
Thethresholdvaluesfortransactionsizescanbeconfiguredgloballyusingthestartupoptions
--rocksdb.intermediate-commit-size:ifthesizeofalloperationsinatransactionreachesthisthreshold,thetransactioniscommittedautomaticallyandanewtransactionisstarted.Thevalueisspecifiedinbytes.
--rocksdb.intermediate-commit-count:ifthenumberofoperationsinatransactionreachesthisvalue,thetransactioniscommittedautomaticallyandanewtransactionisstarted.
--rocksdb.max-transaction-size:thisisanupperlimitforthetotalnumberofbytesofalloperationsinatransaction.Iftheoperationsinatransactionconsumemorethanthisthresholdvalue,thetransactionwillautomaticallyabortwitherror32("resourcelimitexceeded").
Itisalsopossibletooverridethesethresholdspertransaction.
Thefollowingknownissueswillberesolvedinfuturereleases:
theRocksDBengineisnotyetperformance-optimizedandpotentiallynotwellconfigured
collectionsforwhichageoindexispresentwillusecollection-levelwritelocksevenwiththeRocksDBengine.Readsfromthesecollectionscanstillbedoneinparallelbutnowrites
modifyingdocumentsinacollectionwithageoindexwillcausemultipleadditionalwritestoRocksDBformaintainingtheindexstructures
thenumberofdocumentsreportedforcollections(db.<collection>.count())maybeslightlywrongduringtransactionsifthereareparalleltransactionsongoingforthesamecollectionthatalsomodifythenumberofdocuments
theanyoperationtoprovidearandomdocumentfromacollectionissupportedbytheRocksDBenginebuttheoperationhasmuchhigheralgorithmiccomplexitythanintheMMFilesengine.Itisthereforediscouragedtocallitforcasesotherthanmanualinspectionofafewdocumentsinacollection
AQLqueriesintheclusterstillissueanextralockingHTTPrequestpershardthoughthiswouldnotbenecessaryfortheRocksDBengineinmostcases
Installer
KnownIssuesin3.2
497
Upgradingfrom3.1to3.2onWindowsrequirestheusertomanuallycopythedatabasedirectorytothenewlocationandrunanupgradeonthedatabase.PleaseconsulttheDocumentationfordetailedinstructions.
SystemIntegration
OnsomeLinuxsystemssystemdandsystemvmightreportthatthearangodbserviceisingoodconditionwhenitcouldnotbestarted.Inthiscasetheuserneedstocheck/var/log/arangodb3forfurtherinformationaboutthefailedstartup.
MacOSX
Storageengineisnotchangeableonanexistingdatabase.Currentlyonlytheinitialselectionofthestorageengineissupported.Inordertouseanotherstorageengine,youhavetodeleteyourArangoDBapplication(MacApplicationFolder)and/Users/<your_user_name>/Library/ArangoDBfolder.
OpenSSL1.1
ArangoDBhasbeentestedwithOpenSSL1.0onlyandwon'tbuildagainst1.1whencompilingonyourown.SeehereforhowtocompileonsystemsthatshipOpenSSL1.1bydefault.
KnownIssuesin3.2
498
IncompatiblechangesinArangoDB3.2ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB3.2,andadjustanyclientprogramsifnecessary.
AQL
AQLbreakingchangeincluster:TheSHORTEST_PATHstatementusingedge-collectionnamesinsteadofagraphnamenowrequirestoexplicitlynamethevertex-collectionnameswithintheAQLqueryinthecluster.ItcanbedonebyaddingWITH<name>atthebeginningofthequery.
Example:
FORv,eINOUTBOUNDSHORTEST_PATH@startTO@targetedges[...]
Nowhastobe:
WITHvertices
FORv,eINOUTBOUNDSHORTEST_PATH@startTO@targetedges[...]
Thischangeisduetoavoiddead-locksitationsinclusteredcase.Anerrorstatingtheaboveisincluded.
RESTAPIRemovedundocumentedinternalHTTPAPI:
PUT/_api/edgesThedocumentedGET/_api/edgesandtheundocumentedPOST/_api/edgesremainsunmodified.
changeundocumentedbehaviourincaseofinvalidrevisionidsinIf-MatchandIf-None-MatchheadersfromreturningHTTPstatuscode400(badrequest)toreturningHTTPstatuscode412(preconditionfailed).
theRESTAPIforfetchingthelistofcurrentlyrunningAQLqueriesandtheRESTAPIforfetchingthelistofslowAQLqueriesnowreturnanextrabindVarsattributewhichcontainsthebindparametersusedbythequeries.
ThisaffectsthereturnvaluesofthefollowingAPIendpoints:
GET/_api/query/currentGET/_api/query/slow
TheRESTAPIforretrievingindexes(GET/_api/index)nowreturnsthededuplicateattributeforeachindex
TheRESTAPIforcreatingindexes(POST/_api/index)nowacceptstheoptionaldeduplicateattribute
JavaScriptAPI
changeundocumentedbehaviourincaseofinvalidrevisionidsinJavaScriptdocumentoperationsfromreturningerrorcode1239("illegaldocumentrevision")toreturningerrorcode1200("conflict").
thecollection.getIndexes()functionnowreturnsthededuplicateattributeforeachindex
thecollection.ensureIndex()functionnowacceptstheoptionaldeduplicateattribute
Foxx
JWTtokensissuedbythebuilt-inJWTsessionstoragenowcorrectlyspecifytheiatandexpvaluesinsecondsratherthanmillisecondsasspecifiedintheJSONWebTokenstandard.
Incompatiblechangesin3.2
499
Thismayresultinpreviouslyexpiredtokensusingmillisecondsbeingincorrectlyaccepted.ForthisreasonitisrecommendedtoreplacethesigningsecretorsetthenewmaxExpoptiontoareasonablevaluethatissmallerthantheoldestissuedexpirationtimestamp.
ForexamplesettingmaxExpto10**12wouldinvalidateallincorrectlyissuedtokensbefore9September2001withoutimpairingnewtokensuntiltheyear33658(atwhichpointthesetokensarehopefullynolongerrelevant).
ArangoDBrunninginstandalonemodewillcommitallservicesinthejavascript.app-pathtothedatabaseonstartup.ThismayresultinuninstalledservicesshowingupinArangoDBiftheywerenotproperlyremovedfromthefilesystem.
ArangoDBcoordinatorsinaclusternowperformaself-healingstepduringstartuptoensureinstalledservicesareconsistentaccrossallcoordinators.WerecommendbackingupyourservicesandconfigurationbeforeupgradingtoArangoDB3.2,especiallyifyouhavemadeuseofthedevelopmentmode.
Servicesinstalledbeforeupgradingto3.2(includingservicesinstalledonalphareleasesofArangoDB3.2)areNOTpickedupbythecoordinatorself-healingwatchdog.Thiscanbesolvedbyeitherupgrading/replacingtheseservicesorbyusingthe"commit"routeoftheFoxxservicemanagementHTTPAPI,whichcommitstheexactservicesinstalledonagivencoordinatortothecluster.Newserviceswillbepickedupautomatically.
TheformatusedbyFoxxtostoreinternalservicemetadatainthedatabasehasbeensimplifiedandexistingdocumentswillbeupdatedtothenewformat.Ifyouhavemadeanychangestothedatastoredinthe_appssystemcollection,youmaywishtoexportthesechangesastheywillbeoverwritten.
ThereisnowanofficialHTTPAPIformanagingservices.IfyouwerepreviouslyusinganyoftheundocumentedAPIsortheroutesusedbytheadministrativewebinterfacewehighlyrecommendmigratingtothenewAPI.TheoldundocumentedHTTPAPIformananagingservicesisdeprecatedandwillberemovedinafutureversionofArangoDB.
Althoughchangestothefilesystemoutsideofdevelopmentmodewerealreadystronglydiscouraged,thisisareminderthattheyarenolongersupported.Allfilesgeneratedbyservices(whetherbyasetupscriptorduringnormaloperationsuchasuploads)shouldeitherbestoredoutsidetheservicedirectoryorbeconsideredextremelyvolatile.
IntroduceddistinctionbetweenarangoUserandauthorizedinFoxxrequests.ClusterinternalrequestswillneverhaveanarangoUserbutareauthorized.InearlierversionsofArangoDBpartsofthestatisticswerenotaccessiblebythecoordinatorsbecausetheunderlyingFoxxservicecouldn'tauthorizetherequests.Itnowcorrectlychecksthenewreq.authorizedproperty.req.arangoUserstillworksasbefore.Endusersmayusethisnewpropertyaswelltoeasilycheckifarequestisauthorizedornotregardlessofaspecificuser.
Command-lineoptionschanged
--server.maximal-queue-sizeisnowanabsolutemaximum.Ifthequeueisfull,then503isreturned.Settingitto0means"nolimit".Thedefaultvalueforthisoptionisnow0.
thedefaultvaluefor--ssl.protocolhasbeenchangedfrom4(TLSv1)to5(TLSv1.2).
thestartupoptions--database.revision-cache-chunk-sizeand--database.revision-cache-target-sizearenowobsoleteanddonothing
thestartupoption--database.index-threadsoptionisnowobsolete
theoption--javascript.v8-contextsisnowanabsolutemaximum.TheservermaystartlessV8contextsforJavaScriptexecutionatstartup.IfatsomepointtheserverneedsmoreV8contextsitmaystartthemdynamically,untilthenumberofV8contextsreachesthevalueof--javascript.v8-contexts.
theminimumnumberofV8contextstocreateatstartupcanbeconfiguredviathenewstartupoption--javascript.v8-contexts-minimum.
addedcommand-lineoption--javascript.allow-admin-execute
Thisoptioncanbeusedtocontrolwhetheruser-definedJavaScriptcodeisallowedtobeexecutedonserverbysendingviaHTTPtotheAPIendpoint/_admin/executewithanauthenticateduseraccount.Thedefaultvalueisfalse,whichdisablestheexecutionofuser-definedcode.Thisisalsotherecommendedsettingforproduction.Intestenvironments,itmaybeconvenienttoturntheoptiononinordertosendarbitrarysetuporteardowncommandsforexecutionontheserver.
Incompatiblechangesin3.2
500
TheintroductionofthisoptionchangesthedefaultbehaviorofArangoDB3.2:3.2nowbydefaultdisablestheexecutionofJavaScriptcodeviathisAPI,whereasearlierversionsallowedit.Torestoretheoldbehavior,itisnecessarytosettheoptiontotrue.
UsersManagement
Itisnolongersupportedtoaccessthe_userscollecctioninanywaydirectly,exceptthroughtheofficial@arangodb/usersmoduleorthe_apit/usersRESTAPI.
Theaccesstothe_userscollectionfromoutsideofthearangodserverprocessisnowforbidden(Throughdrivers,arangoshortheRESTAPI).Foxxservicesarestillbeabletoaccessthe_userscollectionfornow,butthismightchangeinfutureminorreleases.
Theinternalformatofthedocumentsinthe_userscollectionhaschangedfrompreviousversions
The_queuescollectiononlyallowsread-onlyaccessfromoutsideofthearangodserverprocess.
Accessing_queuesisonlysupportedthroughtheofficial@arangodb/queuesmoduleforFoxxapps.
Incompatiblechangesin3.2
501
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB3.1.ArangoDB3.1alsocontainsseveralbugfixesthatarenotlistedhere.
SmartGraphs
ArangoDB3.1addsafirstmajorenterpriseonlyfeaturecalledSmartGraphs.SmartGraphsformanadditiontothealreadyexistinggraphfeaturesandallowtoscalegraphsbeyondasinglemachinewhilekeepingalmostthesamequeryperformance.TheSmartGraphfeatureissuggestedforallgraphdatabaseusecasesthatrequireaclusterofdatabaseserversforwhateverreason.Youcaneitherhaveagraphthatistoolargetobestoredonasinglemachineonly.Oryoucanhaveasmallgraph,butatthesametimeneedadditionaldatawithhastobeshardedandyouwanttokeepalloftheminthesameenvirenment.Oryousimplyusetheclusterforhigh-availability.InalltheabovecasesSmartGraphswillsignificantlyincreasetheperformanceofgraphoperations.Formoredetailedinformationreadthismanualsection.
Dataformat
Theformatoftherevisionvaluesstoredinthe_revattributeofdocumentshasbeenchangedin3.1.Upto3.0theywerestringscontaininglargishdecimalnumbers.With3.1,revisionvaluesarestillstrings,butareactuallyencodedtimestampsofthecreationdateoftherevisionofthedocument.Thetimestampsareacquiredusingahybridlogicalclock(HLC)ontheDBserverthatholdstherevision(fortheconceptofahybridlogicalclockseethispaper).Seethismanualsectionfordetails.
ArangoDB>=3.1canArangoDB3.0databasedirectoriesandwillsimplycontinuetousetheold_revattributevalues.Newrevisionswillbewrittenwiththenewtimestamps.
ItishighlyrecommendedtobackupallyourdatabeforeloadingadatabasedirectorythatwaswrittenbyArangoDB<=3.0intoanArangoDB>=3.1.
CommunicationLayerArangoDBupto3.0usedlibevforthecommunicationlayer.ArangoDBstartingfrom3.1usesBoostASIO.
StartingwithArangoDB3.1webegintoprovidetheVelocyStreamProtocol(vst)asaadditiontotheestablishedhttpprotocol.
Afewoptionshavechangedconcerningcommunication,pleasecheckoutIncompatiblechangesin3.1.
ClusterForitsinternalclustercommunicationa(bundledversion)ofcurlisnowbeingused.Thisenablesasynchronousoperationthroughouttheclusterandshouldimprovegeneralperformanceslightly.
Authenticationisnowsupportedwithinthecluster.
Documentrevisionscache
TheArangoDBservernowprovidesanin-memorycacheforfrequentlyaccesseddocumentrevisions.Documentsthatareaccessedduringread/writeoperationsareloadedintotherevisionscacheautomatically,andsubsequentlyservedfromthere.
Thecachehasatotaltargetsize,whichcanbecontrolledwiththestartupoption--database.revision-cache-target-size.Oncethecachereachesthetargetsize,olderentriesmaybeevictedfromthecachetofreememory.Notethatthetargetsizecurrentlyisahighwatermarkthatwilltriggercachememorygarbagecollectionifexceeded.However,ifallcachechunksarestillinusewhenthehighwatermarkisreached,thecachemaystillgrowandallocatemorechunksuntilcacheentriesbecomeunusedandareallowedtobegarbage-collected.
WhatsNewin3.1
502
Thecacheismaintainedonaper-collectionbasis,thatis,memoryforthecacheisallocatedonaper-collectionbasisinchunks.Thesizeforthecachememorychunkscanbecontrolledviathestartupoption--database.revision-cache-chunk-size.Thedefaultvalueis4MBperchunk.Biggerchunksizesallowsavingmoredocumentsperchunk,whichcanleadtomoreefficientchunkallocationandlookups,butwillalsoleadtomemorywasteifmanychunksareallocatedandnotfullyused.Thelatterwillbethecaseifthereexistmanysmallcollectionswhichallallocatetheirownchunksbutnotfullyutilizethembecauseofthelownumberofdocuments.
AQL
Functionsadded
ThefollowingAQLfunctionshavebeenaddedin3.1:
OUTERSECTION(array1,array2,...,arrayn):returnsthevaluesthatoccuronlyonceacrossallarraysspecified.
DISTANCE(lat1,lon1,lat2,lon2):returnsthedistancebetweenthetwocoordinatesspecifiedby(lat1,lon1)and(lat2,lon2).Thedistanceiscalculatedusingthehaversineformula.
JSON_STRINGIFY(value):returnsaJSONstringrepresentationofthevalue.
JSON_PARSE(value):convertsaJSON-encodedstringintoaregularobject
Indexusageintraversals
3.1allowsAQLtraversalstouseotherindexesthanjusttheedgeindex.Traversalswithfiltersonedgescannowmakeuseofmorespecificindexes.Forexample,thequery
FORv,e,pIN2OUTBOUND@start@@edge
FILTERp.edges[0].foo=="bar"
RETURN[v,e,p]
mayuseahashindexon["_from","foo"]insteadoftheedgeindexonjust["_from"].
Optimizerimprovements
MaketheAQLqueryoptimizerinjectfilterconditionexpressionsreferredtobyvariablesduringfilterconditionaggregation.Forexample,inthefollowingquery
FORdocINcollection
LETcond1=(doc.value==1)
LETcond2=(doc.value==2)
FILTERcond1||cond2
RETURN{doc,cond1,cond2}
theoptimizerwillnowinjecttheconditionsforcond1andcond2intothefilterconditioncond1||cond2,expandingitto(doc.value==1)||(doc.value==2)andmakingtheseconditionsavailableforindexsearching.
Notethattheoptimizerpreviouslyalreadyinjectedsomeconditionsintootherconditions,butonlyifthevariablethatdefinedtheconditionwasnotusedelsewhere.Forexample,thefilterconditioninthequery
FORdocINcollection
LETcond=(doc.value==1)
FILTERcond
RETURN{doc}
alreadygotoptimizedbeforebecausecondwasonlyusedonceinthequeryandtheoptimizerdecidedtoinjectitintotheplacewhereitwasused.
Thisonlyworkedforvariablesthatwerereferredtoonceinthequery.Whenavariablewasusedmultipletimes,theconditionwasnotinjectedasinthefollowingquery
FORdocINcollection
WhatsNewin3.1
503
LETcond=(doc.value==1)
FILTERcond
RETURN{doc,cond}
3.1allowsusingthisconditionsothatthequerycanuseanindexondoc.value(ifsuchindexexists).
Miscellaneousimprovements
Theperformanceofthe[*]operatorwasimprovedforcasesinwhichthisoperatordidnotuseanyfilters,projectionsand/oroffset/limits.
TheAQLqueryexecutorcannowreportthetimerequiredforloadingandlockingthecollectionsusedinanAQLquery.Whenprofilingisenabled,itwillreportthetotalloadingandlockingtimeforthequeryintheloadingcollectionssub-attributeoftheextra.profilevalueoftheresult.TheloadingandlockingtimecanalsobeviewintheAQLqueryeditorinthewebinterface.
AuditLogAuditlogginghasbeenadded,seeAuditing.
ClienttoolsAddedoption--skip-linesforarangoimpThisallowsskippingthefirstfewlinesfromtheimportfileincasetheCSVorTSVimportareusedandsomeinitiallinesshouldbeskippedfromtheinput.
WebAdminInterfaceTheusabilityoftheAQLeditorsignificantlyimproved.InadditiontothestandardJSONoutput,theAQLEditorisnowabletorenderqueryresultsasagraphprevieworatable.FurthermoretheAQLeditordisplaysqueryprofilinginformation.
AddedanewGraphViewerinordertoexchangethetechnicallyobsoleteversion.ThenewGraphViewerisbasedonCanvasbutdoesalsoincludeafirstWebGLimplementation(limitedfunctionality-willchangeinthefuture).ThenewGraphVieweroffersasmoothwaytodiscoverandvisualizeyourgraphs.
Theshardviewinclustermodenowdisplaysaprogressindicatorwhilemovingshards.
AuthenticationUptoArangoDB3.0authenticationofclientrequestswasonlypossiblewithHTTPbasicauthentication.
Startingwith3.1itisnowpossibletoalsouseaJSONWebTokens(JWT)forauthenticatingincomingrequests.
FordetailschecktheHTTPauthenticationchapter.Bothauthenticationmethodsarevalidandwillbesupportedinthenearfuture.Usewhateversuitsyoubest.
Foxx
GraphQL
ItisnoweasytogetstartedwithprovidingGraphQLAPIsinFoxx,seeFoxxGraphQL.
OAuth2
FoxxnowofficiallyprovidesamoduleforimplementingOAuth2clients,seeFoxxOAuth2.
Per-routemiddleware
WhatsNewin3.1
504
It'snowpossibletospecifymiddlewarefunctionsforaroutewhendefiningaroutehandler.Thesemiddlewarefunctionsonlyapplytothesinglerouteandsharetheroute'sparameterdefinitions.CheckouttheFoxxRouterdocumentationformoreinformation.
WhatsNewin3.1
505
IncompatiblechangesinArangoDB3.1ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB3.1,andadjustanyclientprogramsifnecessary.
CommunicationLayer
TheinternalcommicationlayerisnowbasedonBoostASIO.Afewoptionsregardingthreadsandcommunicationhavebeenchanged.
Therearenolongertwodifferentthreadspools(--scheduler.threadsand--server.threads).Theoption--scheduler.threadshasbeenremoved.Thenumberofthreadsisnowcontrolledbytheoption--server.threadsonly.Bydefault--server.threadsissettothenumberofhyper-cores.
Asaconsequenceofthechange,thefollowing(hidden)startupoptionshavebeenremoved:
--server.extra-threads
--server.aql-threads
--server.backend
--server.show-backends
--server.thread-affinity
AQL
ThebehavioroftheAQLarraycomparisonoperatorshaschangedforemptyarrays:
ALLandANYnowalwaysreturnfalsewhentheleft-handoperandisanemptyarray.Thebehaviorfornon-emptyarraysdoesnotchange:
[]ALL==1willreturnfalse[1]ALL==1willreturntrue[1,2]ALL==1willreturnfalse[2,2]ALL==1willreturnfalse[]ANY==1willreturnfalse[1]ANY==1willreturntrue[1,2]ANY==1willreturntrue[2,2]ANY==1willreturnfalse
NONEnowalwaysreturnstruewhentheleft-handoperandisanemptyarray.Thebehaviorfornon-emptyarraysdoesnotchange:
[]NONE==1willreturntrue[1]NONE==1willreturnfalse[1,2]NONE==1willreturnfalse[2,2]NONE==1willreturntrue
WITHinclustertraversalsisnowmandatoryinordertoavoiddeadlocks.
Dataformatchanges
TheattributemaximalSizehasbeenrenamedtojournalSizeincollectionmeta-datafiles("parameter.json").FilescontainingthemaximalSizeattributewillstillbepickedupcorrectlyfornot-yetadjustedcollections.
Theformatoftherevisionvaluesstoredinthe_revattributeofdocumentshasbeenchangedin3.1.Upto3.0theywerestringscontaininglargishdecimalnumbers.With3.1,revisionvaluesarestillstrings,butareactuallyencodedtimestampsofthecreationdateoftherevisionofthedocument.Thetimestampsareacquiredusingahybridlogicalclock(HLC)ontheDBserverthatholdstherevision(fortheconceptofahybridlogicalclockseethispaper).Seethismanualsectionfordetails.
Incompatiblechangesin3.1
506
ArangoDB>=3.1canArangoDB3.0databasedirectoriesandwillsimplycontinuetousetheold_revattributevalues.Newrevisionswillbewrittenwiththenewtimestamps.
ItishighlyrecommendedtobackupallyourdatabeforeloadingadatabasedirectorythatwaswrittenbyArangoDB<=3.0intoanArangoDB>=3.1.
Tochangeallyourold_revattributesintonewstyletimestampsyouhavetousearangodumptodumpalldataout(usingArangoDB3.0),andusearangorestoreintothenewArangoDB3.1,whichisthesafestwaytoupgrade.
Thechangealsoaffectsthereturnformatof_revvaluesandotherrevisionvaluesinHTTPAPIs(seebelow).
HTTPAPIchanges
APIsadded
ThefollowingHTTPRESTAPIshavebeenaddedforonlinelogleveladjustmentoftheserver:
GET/_admin/log/levelreturnsthecurrentloglevelsettingsPUT/_admin/log/levelmodifiesthecurrentloglevelsettings
APIschanged
thefollowingRESTAPIsthatreturnrevisionidsnowmakeuseofthenewrevisionidformatintroducedin3.1.Allrevisionidsreturnedwillbestringsasin3.0,buthaveadifferentinternalformat.
ThefollowingAPIsareaffected:
GET/_api/collection/{collection}/checksum:revisionattributeGET/_api/collection/{collection}/revision:revisionattributeallotherAPIsthatreturndocuments,whichmayincludethedocuments'_revattribute
Clientapplicationsshouldnottrytointerprettheinternalsofrevisionvalues,butonlyuserevisionvaluesforcheckingwhethertworevisionstringsareidentical.
thereplicationRESTAPIswillnowusetheattributenamejournalSizeinsteadofmaximalSizewhenreturninginformationaboutcollections.
thedefaultvalueforkeepNullhasbeenchangedfromfalsetotrueforthefollowingpartialupdateoperationsforverticesandedgesin/_api/gharial:
PATCH/_api/gharial/{graph}/vertex/{collection}/{key}PATCH/_api/gharial/{graph}/edge/{collection}/{key}
ThevalueforkeepNullcanstillbesetexplicitlytofalsebysettingtheURLparameterkeepNulltoavalueoffalse.
theRESTAPIfordroppingcollections(DELETE/_api/collection)nowacceptsanoptionalquerystringparameterisSystem,whichcansettotrueinordertodropsystemcollections.Iftheparameterisnotsetornotsettotrue,theRESTAPIwillrefusetodropsystemcollections.InpreviousversionsofArangoDB,theisSystemparameterdidnotexist,andtherewasnodistinctionbetweensystemandnon-systemcollectionswhendroppingcollections.
theRESTAPIforretrievingAQLqueryresults(POST/_api/cursor)willnowreturnanadditionalsub-attributeloadingcollectionsthatwillcontainthetotaltimerequiredforloadingandlockingcollectionsduringtheAQLquerywhenprofilingisenabled.Theattributecanbefoundintheextraresultattributeinsub-attributeloadingcollections.Theattributewillonlybesetifprofilingwasenabledforthequery.
FoxxTesting
TheQUnitinterfacetoMochahasbeenremoved.Thisaffectsthebehaviourofthesuite,test,before,after,beforeEachandafterEachfunctionsinFoxxtestsuites.ThesuiteandtestfunctionsarenowprovidedbytheTDDinterface.Thebefore,after,beforeEachandafterEachfunctionsarenowprovidedbytheBDDinterface.
Thisshouldnotcauseanyproblemswithexistingtestsbutmayresultinfailuresintestcasesthatpreviouslypassedforthewrongreasons.Specificallytheexecutionorderofthebefore,after,etcfunctionsnowfollowstheintendedorderandisnolongerarbitrary.
Incompatiblechangesin3.1
507
FordetailsontheexpectedbehaviourofthesefunctionsseethetestingchapterintheFoxxdocumentation.
Incompatiblechangesin3.1
508
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB3.0.ArangoDB3.0alsocontainsseveralbugfixesthatarenotlistedhere.
Internaldataformatchanges
ArangoDBnowusesVelocyPackforstoringdocuments,queryresultsandtemporarilycomputedvalues.Usingasingledataformatremovedtheneedforsomedataconversionsinthecorethatslowedoperationsdownpreviously.
TheVelocyPackformatisalsoquitecompact,andreducesstoragespacerequirementsfor"small"valuessuchasboolean,integers,shortstrings.ThiscanspeedupseveraloperationsinsideAQLqueries.
VelocyPackdocumententriesstoredondiskarealsoself-contained,inthesensethateachstoreddocumentwillcontainallofitsdatatypeandattributenamedescriptions.Whilethismayrequireabitmorespaceforstoringthedocuments,itremovestheoverheadoffetchingattributenamesanddocumentlayoutfromsharedstructuresasinpreviousversionsofArangoDB.Italsosimplifiesthecodepathsforstoringandreadingdocuments.
AQLimprovements
Syntaximprovements
LIKEstring-comparisonoperator
AQLnowprovidesaLIKEoperatorandcanbeusedtocomparestringslikethis,forexampleinsidefilterconditions:
valueLIKEsearch
ThischangemakesLIKEanAQLkeyword.UsingLIKEasanattributeorcollectionnameinAQLthusrequiresquotingthenamefromnowon.
TheLIKEoperatoriscurrentlyimplementedbycallingthealreadyexistingAQLfunctionLIKE,whichalsoremainsoperationalin3.0.UsetheLIKEfunctionincaseyouwanttosearchcase-insensitive(optionalparameter),astheLIKEoperatoralwayscomparescase-sensitive.
AQLarraycomparisonoperators
AllAQLcomparisonoperatorsnowalsoexistinanarrayvariant.Inthearrayvariant,theoperatorisprecededwithoneofthekeywordsALL,ANYorNONE.Usingoneofthesekeywordschangestheoperatorbehaviortoexecutethecomparisonoperationforall,any,ornoneofitslefthandargumentvalues.Itisthereforeexpectedthatthelefthandargumentofanarrayoperatorisanarray.
Examples:
[1,2,3]ALLIN[2,3,4]//false
[1,2,3]ALLIN[1,2,3]//true
[1,2,3]NONEIN[3]//false
[1,2,3]NONEIN[23,42]//true
[1,2,3]ANYIN[4,5,6]//false
[1,2,3]ANYIN[1,42]//true
[1,2,3]ANY==2//true
[1,2,3]ANY==4//false
[1,2,3]ANY>0//true
[1,2,3]ANY<=1//true
[1,2,3]NONE<99//false
[1,2,3]NONE>10//true
[1,2,3]ALL>2//false
[1,2,3]ALL>0//true
[1,2,3]ALL>=3//false
["foo","bar"]ALL!="moo"//true
WhatsNewin3.0
509
["foo","bar"]NONE=="bar"//false
["foo","bar"]ANY=="foo"//true
Regularexpressionstring-comparisonoperators
AQLnowsupportstheoperators=~and!~fortestingstringsagainstregularexpressions.=~testsifastringvaluematchesaregularexpression,and!~testsifastringvaluedoesnotmatcharegularexpression.
Thetwooperatorsexpecttheirleft-handoperandstobestrings,andtheirright-handoperandstobestringscontainingvalidregularexpressionsasspecifiedbelow.
Theregularexpressionsmayconsistofliteralcharactersandthefollowingcharactersandsequences:
.–thedotmatchesanysinglecharacterexceptlineterminators.Toincludelineterminators,use[\s\S]insteadtosimulate.withDOTALLflag.\d–matchesasingledigit,equivalentto[0-9]\s–matchesasinglewhitespacecharacter\S–matchesasinglenon-whitespacecharacter\t–matchesatabcharacter\r–matchesacarriagereturn\n–matchesaline-feedcharacter[xyz]–setofcharacters.matchesanyoftheenclosedcharacters(i.e.x,yorzinthiscase[^xyz]–negatedsetofcharacters.matchesanyothercharacterthantheenclosedones(i.e.anythingbutx,yorzinthiscase)[x-z]–rangeofcharacters.Matchesanyofthecharactersinthespecifiedrange,e.g.[0-9A-F]tomatchanycharacterin0123456789ABCDEF[^x-z]–negatedrangeofcharacters.Matchesanyothercharacterthantheonesspecifiedintherange(xyz)–definesandmatchesapatterngroup(x|y)–matcheseitherxory –matchesthebeginningofthestring(e.g. xyz)$–matchestheendofthestring(e.g.xyz$)
Notethatthecharacters.,*,?,[,],(,),{,}, ,and$haveaspecialmeaninginregularexpressionsandmayneedtobeescapedusingabackslash(\\).Aliteralbackslashshouldalsobeescapedusinganotherbackslash,i.e.\\\\.
Charactersandsequencesmayoptionallyberepeatedusingthefollowingquantifiers:
x*–matcheszeroormoreoccurrencesofxx+–matchesoneormoreoccurrencesofxx?–matchesoneorzerooccurrencesofxx{y}–matchesexactlyyoccurrencesofxx{y,z}–matchesbetweenyandzoccurrencesofxx{y,}–matchesatleastyoccurencesofx
Enclosingidentifiersinforwardticks
AQLidentifierscannowoptionallybeenclosedinforwardticksinadditiontousingbackwardticks.ThisallowsconvenientwritingofAQLqueriesinJavaScripttemplatestrings(whicharedelimitedwithbackticksthemselves),e.g.
varq=`FORdocIN´collection´RETURNdoc.´name´`;
Functionsadded
ThefollowingAQLfunctionshavebeenaddedin3.0:
REGEX_TEST(value,regex):testswhetherthestringvaluematchestheregularexpressionspecifiedinregex.Returnstrueifitmatches,andfalseotherwise.
Thesyntaxforregularexpressionsisthesameasfortheregularexpressionoperators=~and!~.
WhatsNewin3.0
510
HASH(value):Calculatesahashvalueforvalue.valueisnotrequiredtobeastring,butcanhaveanydatatype.Thecalculatedhashvaluewilltakethedatatypeofvalueintoaccount,soforexamplethenumber1andthestring"1" willhavedifferenthashvalues.Forarraysthehashvalueswillbecrearedifthearrayscontainexactlythesamevalues(includingvaluetypes)inthesameorder.Forobjectsthesamehashvalueswillbecreatediftheobjectshaveexactlythesameattributenamesandvalues(includingvaluetypes).Theorderinwhichattributesappearinsideobjectsisnotimportantforhashing.Thehashvaluereturnedbythisfunctionisanumber.ThehashalgorithmisnotguaranteedtoremainthesameinfutureversionsofArangoDB.Thehashvaluesshouldthereforebeusedonlyfortemporarycalculations,e.g.tocompareiftwodocumentsarethesame,orforgroupingvaluesinqueries.
TYPENAME(value):Returnsthedatatypenameofvalue.Thedatatypenamecanbeeithernull,bool,number,string,arrayorobject.
LOG(value):Returnsthenaturallogarithmofvalue.ThebaseisEuler'sconstant(2.71828...).
LOG2(value):Returnsthebase2logarithmofvalue.
LOG10(value):Returnsthebase10logarithmofvalue.
EXP(value):ReturnsEuler'sconstant(2.71828...)raisedtothepowerofvalue.
EXP2(value):Returns2raisedtothepowerofvalue.
SIN(value):Returnsthesineofvalue.
COS(value):Returnsthecosineofvalue.
TAN(value):Returnsthetangentofvalue.
ASIN(value):Returnsthearcsineofvalue.
ACOS(value):Returnsthearccosineofvalue.
ATAN(value):Returnsthearctangentofvalue.
ATAN2(y,x):Returnsthearctangentofthequotientofyandx.
RADIANS(value):Returnstheangleconvertedfromdegreestoradians.
DEGREES(value):Returnstheangleconvertedfromradianstodegrees.
Optimizerimprovements
"inline-subqueries"rule
TheAQLoptimizerrule"inline-subqueries"hasbeenadded.ThisrulecanpulloutcertainsubqueriesthatareusedasanoperandtoaFORlooponelevelhigher,eliminatingthesubquerycompletely.Thisreducescomplexityofthequery'sexecutionplanandwilllikelyenablefurtheroptimizations.Forexample,thequery
FORiIN(
FORjIN[1,2,3]
RETURNj
)
RETURNi
willbetransformedbytheruleto:
FORiIN[1,2,3]
RETURNi
Thequery
FORnameIN(
FORdocIN_users
FILTERdoc.status==1
RETURNdoc.name
)
LIMIT2
WhatsNewin3.0
511
RETURNname
willbetransformedinto
FORtmpIN_users
FILTERtmp.status==1
LIMIT2
RETURNtmp.name
TherulewillonlyfirewhenthesubqueryisusedasanoperandtoaFORloop,andifthesubquerydoesnotcontainaCOLLECTwithanINTOvariable.
"remove-unnecessary-calculations"rule
TheAQLoptimizerrule"remove-unnecessary-calculations"nowfiresinmorecasesthaninpreviousversions.Thisruleremovescalculationsfromexecutionplans,andbyhavinglesscalculationsdone,aquerymayexecutefasterorrequireslessmemory.
Therulewillnowremovecalculationsthatareusedexactlyonceinotherexpressions(e.g.LETa=docRETURNa.value)andcalculations,orcalculationsthatarejustreferencestoothervariables(e.g.LETa=b).
"optimize-traversals"rule
TheAQLoptimizerrule"merge-traversal-filter"wasrenamedto"optimize-traversals".TherulewillremoveunusededgeandpathresultvariablesfromthetraversalincasetheyarespecifiedintheFORsectionofthetraversal,butnotreferencedlaterinthequery.Thissavesconstructingedgesandpathsresultsthatarenotusedlater.
AQLnowusesVelocyPackinternallyforstoringintermediatevalues.Formanyvaluetypesitcannowgetawaywithoutextramemoryallocationsandlessinternalconversions.ValuescanbepassedintointernalAQLfunctionswithoutcopyingthem.ThiscanleadtoreducedqueryexecutiontimesforqueriesthatuseC++-basedAQLfunctions.
"replace-or-with-in"and"use-index-for-sort"rules
Theserulesnowfireinsomeadditionalcases,whichallowssimplifyingindexlookupconditionsandremovingSortNodesfromexecutionplans.
Clusterstatemanagement
Thecluster'sinternalstateinformationisnowalsomanagedbyArangoDBinstances.Earlierversionsreliedonthirdpartysoftwarebeinginstalledforthestoringtheclusterstate.ThestateismanagedbydedicatedArangoDBinstances,whichcanbestartedinaspecialagencymode.Theseinstancescanoperateinadistributedfashion.Theywillautomaticallyelectoneofthemtobecometheirleader,beingresponsibileforstoringthestatechangessentfromserversinthecluster.Theotherinstanceswillautomaticallyfollowtheleaderandwilltransparentlystandinshoulditbecomeunavailable.Theagencyinstancesarealsoself-organizing:theywillcontinuouslyprobeeachotherandre-electleaders.Thecommunicationbetweentheagencyinstancesusetheconsensus-basedRAFTprotocol.
TheoperationsforstoringandretrievingclusterstateinformationarenowmuchlessexpensivefromanArangoDBclusternodeperspective,whichinturnallowsforfasterclusteroperationsthatneedtofetchorupdatetheoverallclusterstate.
_fromand _toattributesofedgesareupdatableandusableinindexes
InArangoDBpriorto3.0theattributes_fromand_toofedgesweretreatedspeciallywhenloadingorstoringedges.Thatspecialhandlingledtotheseattributesbeingnotasflexibleasregulardocumentattributes.Forexample,the_fromand_toattributevaluesofanexistingedgecouldnotbeupdatedoncetheedgewascreated.Nowthisispossibleviathesingle-documentAPIsandviaAQL.
Additionally,the_fromand_toattributescouldnotbeindexedinuser-definedindexes,e.g.tomakeeachcombinationof_fromand_tounique.Finally,as_fromand_toreferencedthelinkedcollectionsbycollectionidandnotbycollectionname,theirmeaningbecameunclearonceareferencedcollectionwasdropped.Thecollectionidstoredinedgesthenbecameunusable,andwhenaccessing
WhatsNewin3.0
512
suchedgethecollectionnamepartofitwasalwaystranslatedto_undefined.
InArangoDB3.0,the_fromand_tovaluesofedgesaresavedasregularstrings.Thisallowsusing_fromand_toinuser-definedindexes.Additionally,thisallowstoupdatethe_fromand_tovaluesofexistingedges.Furthermore,collectionsreferencedby_fromand_tovaluesmaybedroppedandre-createdlater.Any_fromand_tovaluesofedgespointingtosuchdroppedcollectionareunaffectedbythedropoperationnow.
UnifiedAPIsforCRUDoperations
TheCRUDAPIsfordocumentsandedgehavebeenunified.EdgescannowbeinsertedandmodifiedviathesameAPIsasdocuments._fromand_toattributevaluescanbepassedasregulardocumentattributesnow:
db.myedges.insert({_from:"myvertices/some",_to:"myvertices/other",...});
Passing_fromand_toseparatelyasitwasrequiredinearlierversionsisnotnecessaryanymorebutwillstillwork:
db.myedges.insert("myvertices/some","myvertices/other",{...});
TheCRUDoperationsnowalsosupportbatchvariantsthatworksonarraysofdocuments/edges,e.g.
db.myedges.insert([
{_from:"myvertices/some",_to:"myvertices/other",...},
{_from:"myvertices/who",_to:"myvertices/friend",...},
{_from:"myvertices/one",_to:"myvertices/two",...},
]);
ThebatchvariantsarealsoavailableinArangoDB'sHTTPAPI.Theycanbeusedtomoreefficientlycarryoutoperationswithmultipledocumentsthantheirsingle-documentequivalents,whichrequiredoneHTTPrequestperoperation.Withthebatchoperations,theHTTPrequest/responseoverheadcanbeamortizedacrossmultipleoperations.
Persistentindexes
ArangoDB3.0providesanexperimentalpersistentindexfeature.Persistentindexesstoretheindexvaluesondiskinsteadofin-memoryonly.Thismeanstheindexesdonotneedtoberebuiltin-memorywhenacollectionisloadedorreloaded,whichshouldimprovecollectionloadingtimes.
ThepersistentindexesinArangoDBarebasedontheRocksDBengine.Tocreateapersistentindexforacollection,createanindexoftype"rocksdb"asfollows:
db.mycollection.ensureIndex({type:"rocksdb",fields:["fieldname"]});
Thepersistentindexesaresorted,sotheyallowequalitylookupsandrangequeries.Notethatthefeatureisstillhighlyexperimentalandhassomeknowndeficiencies.Itwillbefinalizeduntilthereleaseofthe3.0stableversion.
UpgradedV8version
TheV8enginethatisusedinsideArangoDBtoexecuteJavaScriptcodehasbeenupgradedfromversion4.3.61to5.0.71.39.ThenewversionmakesseveralmoreES6featuresavailablebydefault,including
arrowfunctionscomputedpropertynamesrestparametersarraydestructuringnumericandobjectliterals
WhatsNewin3.0
513
WebAdminInterface
TheArangoDB3.0webinterfaceissignificantlyimproved.Itnowcomeswithamoreresponsivedesign,makingiteasiertouseondifferentdevices.Navigationandmenushavebeensimplified,andrelateditemshavebeenregroupedtostayclosertogetherandallowtighterworkflows.
TheAQLqueryeditorisnowmucheasiertouse.Multiplequeriescanbestartedandtrackedinparallel,whileresultsofearlierqueriesarestillpreserved.Queriesstillrunningcanbecanceleddirectlyfromtheeditor.TheAQLqueryeditornowallowstheusageofbindparameterstoo,andprovidesahelperforfindingcollectionnames,AQLfunctionnamesandkeywordsquickly.
Thewebinterfacenowkeepstrackofwhethertheserverisofflineandofwhichserver-sideoperationshavebeenstartedandarestillrunning.Itnowremainsusablewhilesuchlonger-runningoperationsareongoing.Italsokeepsmorestateaboutuser'schoices(e.g.windowssizes,whetherthetreeorthecodeviewwaslastusedinthedocumenteditor).
Clusterstatisticsarenowintegratedintothewebinterfaceaswell.Additionally,amenuitem"Helpus"hasbeenaddedtoeasilyprovidetheArangoDBteamfeedbackabouttheproduct.
Thefrontendmaynowbemountedbehindareverseproxyonadifferentpath.ForthistoworktheproxyshouldsendaX-Script-Nameheadercontainingthepath.
Abackendconfigurationforhaproxymightlooklikethis:
reqaddX-Script-Name:\/arangodb
Thefrontendwillrecognizethesubpathandproduceappropriatelinks.ArangoDBwillonlyacceptpathsfromtrustedfrontendproxies.Trustedproxiesmaybeaddedonstartup:
--frontend.proxy-request-checktrue--frontend.trusted-proxy192.168.1.117
--frontend.trusted-proxymaybeanyaddressornetmask.
Todisablethecheckandblindlyacceptanyx-script-nameset--frontend.proxy-request-checktofalse.
Foxximprovements
TheFoxxframeworkhasbeencompletelyrewrittenfor3.0withanew,simplerandmorefamiliarAPI.Themostnotablechangesare:
Legacymodefor2.8services
Stuckwitholdcode?Youcancontinueusingyour2.8-compatibleFoxxserviceswith3.0byadding"engines":{"arangodb":"^2.8.0"}(orsimilarversionrangesthatexclude3.0andup)totheservicemanifest.
Nomoreglobalvariablesandmagicalcomments
TheapplicationContextisnowmodule.context.Insteadofmagicalcommentsjustusethesummaryanddescriptionmethodstodocumentyourroutes.
RepositoryandModelhavebeenremoved
InsteadofrepositoriesjustuseArangoDBcollectionsdirectly.Forvalidationsimplyusethejoischemas(butwrappedinjoi.object())thatpreviouslylivedinsidethemodel.CollectionsandqueriesreturnplainJavaScriptobjects.
Controllershavebeenreplacedwithnestablerouters
Createrouterswithrequire('@arangodb/foxx/router')(),attachthemtoyourservicewithmodule.context.use(router).Becauseroutersarenolongermountedautomagically,youcanexportandimportthemlikeanyotherobject.Userouter.use('/path',subRouter)tonestroutersasdeeplyasyouwant.
Routescanbenamedandreversed
NomorememorizingURLs:addanametoyourroutelikerouter.get('/hello/:name',function(){...},'hello')andredirecttothefullURLwithres.redirect(req.resolve('hello',{name:'world'})).
WhatsNewin3.0
514
Simplerexpress-likemiddleware
Ifyoualreadyknowexpress,thisshouldbefamiliar.Here'sarequestloggerinthreelinesofcode:
router.use(function(req,res,next){
varstart=Date.now();
try{next();}
finally{console.log(`${req.method}${req.url}${res.statusCode}${Date.now()-start}ms`);}
});
Sessionsandauthwithoutdependencies
Tomakeiteasiertogetstarted,thefunctionalitypreviouslyprovidedbythesimple-auth,oauth2,sessions-localandsessions-jwtserviceshavebeenmovedintoFoxxasthe@arangodb/foxx/auth,@arangodb/foxx/oauth2and@arangodb/foxx/sessionsmodules.
Logging
ArangoDB'sloggingisnowgroupedintotopics.Thelogverbosityandoutputfilescanbeadjustedperlogtopic.Forexample
--log.levelstartup=trace--log.levelqueries=trace--log.levelinfo
willlogmessagesconcerningstartupattracelevel,AQLqueriesattracelevelandeverythingelseatinfolevel.--log.levelcanbespecifiedmultipletimesatstartup,forasmanytopicsasneeded.
Somerelevantlogtopicsavailablein3.0are:
collector:informationabouttheWALcollector'sstatecompactor:informationaboutthecollectiondatafilecompactordatafiles:datafile-relatedoperationsmmap:informationaboutmemory-mappingoperations(includingmsync)queries:executedAQLqueries,slowqueriesreplication:replication-relatedinforequests:HTTPrequestsstartup:informationaboutserverstartupandshutdownthreads:informationaboutthreads
Thisalsoallowsdirectinglogoutputtodifferentfilesbasedontopics.Forexample,tologallAQLqueriestoafile"queries.log"onecanusetheoptions:
--log.levelqueries=trace--log.outputqueries=file:///path/to/queries.log
ToadditionallylogHTTPrequesttoafilenamed"requests.log"addtheoptions:
--log.levelrequests=info--log.outputrequests=file:///path/to/requests.log
Buildsystem
ArangoDBnowusesthecross-platformbuildsystemCMakeforallitsbuilds.Previousversionsusedtwodifferentbuildsystems,makingdevelopmentandcontributionsharderthannecessary.Nowthebuildsystemisunified,andalltargets(Linux,Windows,MacOS)arebuiltfromthesamesetofbuildinstructions.
Documentation
Thedocumentationhasbeenenhancedandre-organizedtobemoreintuitive.
WhatsNewin3.0
515
AnewintroductionforbeginnersshouldbringyouuptospeedwithArangoDBinlessthananhour.Additionaltopicshavebeenintroducedandwillbeextendedwithupcomingreleases.
ThetopicsAQLandHTTPAPIarenowseparatedfromthemanualforbettersearchabilityandlessconfusion.Aversionswitchermakesiteasiertojumptotheversionofthedocsyouareinterestedin.
WhatsNewin3.0
516
IncompatiblechangesinArangoDB3.0ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB3.0,andadjustanyclientprogramsifnecessary.
Buildsystem
BuildingArangoDB3.0fromsourcenowrequiresCMake.
Thepre-3.0buildsystemusedaconfigure-basedapproach.ThestepstobuildArangoDB2.8fromsourcecodewere:
makesetup
./configure<options>
make
Thesestepswillnotworkanymore,asArangoDB3.0doesnotcomewithaconfigurescript.
Tobuild3.0onLinux,createaseparatebuilddirectoryfirst:
mkdir-pbuild
andthencreatetheinitialbuildscriptsonceusingCMake:
(cdbuild&&cmake<options>..)
Theabovecommandwillconfigurethebuildandcheckfortherequireddependencies.Ifeverythingworkswelltheactualbuildcanbestartedwith
(cdbuild&&make)
ThebinariesfortheArangoDBserverandallclienttoolswillthenbecreatedinsidethebuilddirectory.TostartArangoDBlocallyfromthebuilddirectory,use
build/bin/arangod<options>
Datafilesanddatafilenames
ArangoDB3.0usesanewVelocyPack-basedformatforstoringdatainWALlogfilesandcollectiondatafiles.ThefileformatisnotcompatiblewiththefilesusedinpriorversionsofArangoDB.ThatmeansdatafileswrittenbyArangoDB3.0cannotbeusedinearlierversionsandviceversa.
Thepatternforcollectiondirectorynameswaschangedin3.0toincludearandomidcomponentattheend.Thenewpatterniscollection-<id>-<random>,where<id>isthecollectionidand<random>isarandomnumber.PreviousversionsofArangoDBusedapatterncollection-<id>withouttherandomnumber.
UserManagementUnlikeArangoDB2.x,ArangoDB3.0usersarenowseparatedfromdatabases,andyoucangrantoneormoredatabasepermissionstoauser.
IfyouwanttomimicthebehaviorofArangoDB,youshouldnameyouruserslikeusername@dbname.
Usersthatcanaccessthe_systemdatabaseareallowedtomanageusersandpermissionsforalldatabases.
Incompatiblechangesin3.0
517
Edgesandedgesattributes
InArangoDBpriorto3.0theattributes_fromand_toofedgesweretreatedspeciallywhenloadingorstoringedges.Thatspecialhandlingledtotheseattributesbeingnotasflexibleasregulardocumentattributes.Forexample,the_fromand_toattributevaluesofanexistingedgecouldnotbeupdatedoncetheedgewascreated.Additionally,the_fromand_toattributescouldnotbeindexedinuser-definedindexes,e.g.tomakeeachcombinationof_fromand_tounique.Finally,as_fromand_toreferencedthelinkedcollectionsbycollectionidandnotbycollectionname,theirmeaningbecameunclearonceareferencedcollectionwasdropped.Thecollectionidstoredinedgesthenbecameunusable,andwhenaccessingsuchedgethecollectionnamepartofitwasalwaystranslatedto_undefined.
InArangoDB3.0,the_fromand_tovaluesofedgesaresavedasregularstrings.Thisallowsusing_fromand_toinuser-definedindexes.Additionallythisallowsupdatingthe_fromand_tovaluesofexistingedges.Furthermore,collectionsreferencedby_fromand_tovaluesmaybedroppedandre-createdlater.Any_fromand_tovaluesofedgespointingtosuchdroppedcollectionareunaffectedbythedropoperationnow.Alsonotethatrenamingthecollectionreferencedin_fromand_toinArangoDB2.8alsorelinkedtheedges.In3.0theedgesareNOTautomaticallyrelinkedtothenewcollectionanymore.
DocumentsDocuments(incontrasttoedges)cannotcontaintheattributes_fromor_toonthemainlevelinArangoDB3.0.Theseattributeswillbeautomaticallyremovedwhensavingdocuments(i.e.non-edges)._fromand_tocanbestillusedinsub-objectsinsidedocuments.
The_fromand_toattributeswillofcoursebepreservedandarestillrequiredwhensavingedges.
AQL
Edgeshandling
WhenupdatingorreplacingedgesviaAQL,anymodificationstothe_fromand_toattributesofedgeswereignoredbypreviousversionsofArangoDB,withoutsignalinganyerrors.Thiswasduetothe_fromand_toattributesbeingimmutableinearlierversionsofArangoDB.
From3.0on,the_fromand_toattributesofedgesaremutable,soanyAQLqueriesthatmodifythe_fromor_toattributevaluesofedgeswillattempttoactuallychangetheseattributes.Clientsshouldbeawareofthischangeandshouldreviewtheirqueriesthatmodifyedgestoruleoutunintendedside-effects.
Additionally,whencompletelyreplacingthedataofexistingedgesviatheAQLREPLACEoperation,itisnowrequiredtospecifyvaluesforthe_fromand_toattributes,asREPLACErequirestheentirenewdocumenttobespecified.Ifeither_fromor_toaremissingfromthereplacementdocument,anREPLACEoperationwillfail.
Graphfunctions
Inversion3.0allformergraphrelatedfunctionshavebeenremovedfromAQLtobereplacedbynativeAQLconstructs.Theseconstructsallowformorefine-grainedfilteringonseveralgraphlevels.AlsothisallowstheAQLoptimizertoautomaticallyimprovethesequeriesbyenhancingthemwithappropriateindexes.Wehavecreatedrecipestoupgradefrom2.8to3.0whenusingthesefunctions.
Thefunctions:
GRAPH_COMMON_NEIGHBORSGRAPH_COMMON_PROPERTIESGRAPH_DISTANCE_TOGRAPH_EDGESGRAPH_NEIGHBORSGRAPH_TRAVERSALGRAPH_TRAVERSAL_TREEGRAPH_SHORTEST_PATHGRAPH_PATHSGRAPH_VERTICES
Incompatiblechangesin3.0
518
arecoveredinMigratingGRAPH_*Functionsfrom2.8orearlierto3.0
GRAPH_ABSOLUTE_BETWEENNESSGRAPH_ABSOLUTE_CLOSENESSGRAPH_ABSOLUTE_ECCENTRICITYGRAPH_BETWEENNESSGRAPH_CLOSENESSGRAPH_DIAMETERGRAPH_ECCENTRICITYGRAPH_RADIUS
arecoveredinMigratingGRAPH_*Measurementsfrom2.8orearlierto3.0
EDGESNEIGHBORSPATHSTRAVERSALTRAVERSAL_TREE
arecoveredinMigratinganonymousgraphfunctionsfrom2.8orearlierto3.0
Typecastingfunctions
ThetypecastingappliedbytheTO_NUMBER()AQLfunctionhaschangedasfollows:
stringvaluesthatdonotcontainavalidnumericvaluearenowconvertedtothenumber0.InpreviousversionsofArangoDBsuchstringvalueswereconvertedtothevaluenull.arrayvalueswithmorethan1memberarenowconvertedtothenumber0.InpreviousversionsofArangoDBsucharrayswereconvertedtothevaluenull.objects/documentsarenowconvertedtothenumber0.InpreviousversionsofArangoDBobjects/documentswereconvertedtothevaluenull.
Additionally,theTO_STRING()AQLfunctionnowconvertsnullvaluesintoanemptystring("")insteadofthestring"null",whichismoreinlinewithLENGTH(null)returning0andnot4sincev2.6.
TheoutputofTO_STRING()hasalsochangedforarraysandobjectsasfollows:
arraysarenowconvertedintotheirJSON-stringifyequivalents,e.g.
[]isnowconvertedto[][1,2,3]isnowconvertedto[1,2,3]["test",1,2]isnowconvertedto["test",1,2]`
PreviousversionsofArangoDBconvertedarrayswithnomembersintotheemptystring,andnon-emptyarraysintoacomma-separatedlistofmembervalues,withoutthesurroundingangularbrackets.Additionally,stringarraymemberswerenotenclosedinquotesintheresultstring:
[]wasconvertedto``[1,2,3]wasconvertedto1,2,3["test",1,2]wasconvertedtotest,1,2`
objectsarenowconvertedtotheirJSON-stringifyequivalents,e.g.
{}isconvertedto{}{a:1,b:2}isconvertedto{"a":1,"b":2}{"test":"foobar"}isconvertedto{"test":"foobar"}
PreviousversionsofArangoDBalwaysconvertedobjectsintothestring[objectObject]
ThischangealsoaffectsotherpartsinAQLthatusedTO_STRING()toimplicitlycastoperandstostrings.ItalsoaffectstheAQLfunctionsCONCAT()andCONCAT_SEPARATOR()whichtreatedarrayvaluesdifferently.PreviousversionsofArangoDBautomaticallyflattenedarrayvaluesinthefirstlevelofthearray,e.g.CONCAT([1,2,3,[4,5,6]])produced1,2,3,4,5,6.Nowthiswillproduce[1,2,3,[4,5,6]].Toflattenarraymembersonthetoplevel,youcannowusethemoreexplicitCONCAT(FLATTEN([1,2,3,[4,5,6]],1)).
Incompatiblechangesin3.0
519
Arithmeticoperators
AsthearithmeticoperationsinAQLimplicitlyconverttheiroperandstonumericvaluesusingTO_NUMBER(),theircastingbehaviorhasalsochangedasdescribedabove.
Someexamplesofthechangedbehavior:
"foo"+1produces1now.Inpreviousversionsthisproducednull.[1,2]+1produces1.Inpreviousversionsthisproducednull.1+"foo"+1´produces2now.Inpreviousversionthisproduced1`.
Attributenamesandparameters
PreviousversionsofArangoDBhadsometroublewithattributenamesthatcontainedthedotsymbol(.).SomecodepartsinAQLusedthedotsymboltosplitanattributenameintosub-components,soanattributenameda.bwasnotcompletelydistinguishablefromanattributeawithasub-attributeb.Thisinconsistentbehaviorsometimesallowed"hacks"toworksuchaspassingsub-attributesinabindparameterasfollows:
FORdocINcollection
FILTERdoc.@name==1
RETURNdoc
Ifthebindparameter@namecontainedthedotsymbol(e.g.@bind=a.b,itwasunclearwhetherthisshouldtriggersub-attributeaccess(i.e.doc.a.b)oraaccesstoanattributewithexactlythespecifiedname(i.e.doc["a.b"]).
ArangoDB3.0nowhandlesattributenamescontainingthedotsymbolproperly,andsendingabindparameter@name=a.bwillnowalwaystriggeranaccesstotheattributedoc["a.b"],notthesub-attributebofaindoc.
Forusersthatusedthe"hack"ofpassingbindparameterscontainingdotsymboltoaccesssub-attributes,ArangoDB3.0allowsspecifyingtheattributenamepartsasanarrayofstrings,e.g.@name=["a","b"],whichwillberesolvedtothesub-attributeaccessdoc.a.bwhenthequeryisexecuted.
Keywords
LIKEisnowakeywordinAQL.UsingLIKEineithercaseasanattributeorcollectionnameinAQLqueriesnowrequiresquoting.
SHORTEST_PATHisnowakeywordinAQL.UsingSHORTEST_PATHineithercaseasanattributeorcollectionnameinAQLqueriesnowrequiresquoting.
Subqueries
Queriesthatcontainsubqueriesthatcontaindata-modificationoperationssuchasINSERT,UPDATE,REPLACE,UPSERTorREMOVEwillnowrefusetoexecuteifthecollectionaffectedbythesubquery'sdata-modificationoperationisread-accessedinanouterscopeofthequery.
Forexample,thefollowingquerywillrefusetoexecuteasthecollectionmyCollectionismodifiedinthesubquerybutalsoread-accessedintheouterscope:
FORdocINmyCollection
LETchanges=(
FORwhatINmyCollection
FILTERwhat.value==1
REMOVEwhatINmyCollection
)
RETURNdoc
Itisstillpossibletowritetocollectionsfromwhichdataisreadinthesamequery,e.g.
FORdocINmyCollection
FILTERdoc.value==1
REMOVEdocINmyCollection
Incompatiblechangesin3.0
520
andtomodifydataindifferentcollectionviasubqueries.
Otherchanges
TheAQLoptimizerrule"merge-traversal-filter"thatalreadyexistedin3.0wasrenamedto"optimize-traversals".ThisshouldbeofnorelevancetoclientapplicationsexceptiftheyprogramaticallylookforappliedoptimizerrulesintheexplainoutofAQLqueries.
TheorderofresultscreatedbytheAQLfunctionsVALUES()andATTRIBUTES()wasneverguaranteedanditonlyhadthe"correct"orderingbyaccidentwheniteratingoverobjectsthatwerenotloadedfromthedatabase.Assomeofthefunctioninternalshavechanged,the"correct"orderingwillnotappearanymore,andstillnoresultorderisguaranteedbythesefunctionsunlessthesortparameterisspecified(fortheATTRIBUTES()function).
UpgradedV8version
TheV8enginethatisusedinsideArangoDBtoexecuteJavaScriptcodehasbeenupgradedfromversion4.3.61to5.0.71.39.Thenewversionshouldbemostlycompatibletotheoldversion,buttheremaybesubtledifferences,includingchangesoferrormessagetextsthrownbytheengine.Furthermore,someV8startupparametershavechangedtheirmeaningorhavebeenremovedinthenewversion.ThisisonlyrelevantwhenArangoDBorArangoShellarestartedwithacustomvalueforthe--javascript.v8-optionsstartupoption.
Amongothers,thefollowingV8optionschangeinthenewversionofArangoDB:
--es_staging:in2.8ithadthemeaningenableallcompletedharmonyfeatures,in3.0theoptionmeansenabletest-worthyharmonyfeatures(forinternaluseonly)
--strong_this:thisoptionwasn'tpresentin2.8.In3.0itmeansdon'tallow'this'toescapefromconstructorsanddefaultstotrue.
--harmony_regexps:thisoptionsmeansenable"harmonyregularexpressionextensions"andchangesitsdefaultvaluefromfalsetotrue
--harmony_proxies:thisoptionsmeansenable"harmonyproxies"andchangesitsdefaultvaluefromfalsetotrue
--harmony_reflect:thisoptionsmeansenable"harmonyReflectAPI"andchangesitsdefaultvaluefromfalsetotrue
--harmony_sloppy:thisoptionsmeansenable"harmonyfeaturesinsloppymode"andchangesitsdefaultvaluefromfalsetotrue
--harmony_tostring:thisoptionsmeansenable"harmonytoString"andchangesitsdefaultvaluefromfalsetotrue
--harmony_unicode_regexps:thisoptionsmeansenable"harmonyunicoderegexps"andchangesitsdefaultvaluefromfalsetotrue
--harmony_arrays,--harmony_array_includes,--harmony_computed_property_names,--harmony_arrow_functions,--harmony_rest_parameters,--harmony_classes,--harmony_object_literals,--harmony_numeric_literals,--harmony_unicode:theseoptionhavebeenremovedinV85.
AsaconsequenceoftheupgradetoV8version5,theimplementationoftheJavaScriptBufferobjecthadtobechanged.JavaScriptBufferobjectsinArangoDBnowalwaysstoretheirdataontheheap.ThereisnosharedpoolforsmallBuffervalues,andnopointingintoexistingBufferdatawhenextractingslices.ThischangemayincreasethecostofcreatingBufferswithshortcontentsorwhenpeekingintoexistingBuffers,butwasrequiredforsafermemorymanagementandtopreventleaks.
JavaScriptAPIchanges
ThefollowingincompatiblechangeshavebeenmadetotheJavaScriptAPIinArangoDB3.0:
Foxx
TheFoxxframeworkhasbeencompletelyrewrittenfor3.0withanew,simplerandmorefamiliarAPI.TomakeFoxxservicesdevelopedfor2.8orearlierArangoDBversionsrunin3.0,theservice'smanifestfileneedstobeedited.
ToenablethelegacymodeforaFoxxservice,add"engines":{"arangodb":"^2.8.0"}(orsimilarversionrangesthatexclude3.0andup)totheservicemanifestfile(named"manifest.json",locatedintheservice'sbasedirectory).
Incompatiblechangesin3.0
521
Require
ModulesshippedwithArangoDBcannowberequiredusingthepattern@arangodb/<module>insteadoforg/arangodb/<module>,e.g.
varcluster=require("@arangodb/cluster");
Theoldformatcanstillbeusedforcompatibility:
varcluster=require("org/arangodb/cluster");
ArangoDBpriortoversion3.0allowedatransparentuseofCoffeeScriptsourcefileswiththerequire()function.FileswithafilenameextensionofcoffeewereautomaticallysentthroughaCoffeeScriptparserandtranspiledintoJavaScripton-the-fly.ThissupportisgonewithArangoDB3.0.TorunanyCoffeeScriptsourcefiles,theymustbeconvertedtoJavaScriptbytheclientapplication.
Responseobject
The@arangodb/requestresponseobjectnowstorestheparsedJSONresponsebodyinapropertyjsoninsteadofbodywhentherequestwasmadeusingthejsonoption.Thebodyinsteadcontainstheresponsebodyasastring.
EdgesAPI
Whencompletelyreplacinganedgeviaacollection'sreplace()functionthereplacingedgedatanowneedstocontainthe_fromand_toattributesforthenewedge.PreviousversionsofArangoDBdidnotrequiretheedgedatatocontain_fromand_toattributeswhenreplacinganedge,since_fromand_tovalueswereimmutableforexistingedges.
Forexample,thefollowingcallworkedinArangoDB2.8butwillfailin3.0:
db.edgeCollection.replace("myKey",{value:"test"});
TomakethisworkinArangoDB3.0,_fromand_toneedtobeaddedtothereplacementdata:
db.edgeCollection.replace("myKey",{_from:"myVertexCollection/1",_to:"myVertexCollection/2",value:"test"});
Notethatthisonlyaffectsthereplace()functionbutnotupdate(),whichwillonlyupdatethespecifiedattributesoftheedgeandleaveallothersintact.
Additionally,thefunctionsedges(),outEdges()andinEdges()withanarrayofedgeidswillnowmaketheedgeidsuniquebeforereturningtheconnectededges.Thisisprobablydesiredanyway,asresultswillbereturnedonlyonceperdistinctinputedgeid.However,itmaybreakclientapplicationsthatrelyontheoldbehavior.
DatabasesAPI
The_listDatabases()functionofthedbobjecthasbeenrenamedto_databases(),makingitconsistentwiththe_collections()function.Alsothe_listEndpoints()functionhasbeenrenamedto_endpoints().
CollectionAPI
Examplematching
ThecollectionfunctionbyExampleHash()andbyExampleSkiplist()havebeenremovedin3.0.Theirfunctionalityisprovidedbycollection'sbyExample()function,whichwillautomaticallyuseasuitableindexifpresent.
ThecollectionfunctionbyConditionSkiplist()hasbeenremovedin3.0.ThesamefunctionalitycanbeachievedbyissuinganAQLquerywiththetargetcondition,whichwillautomaticallyuseasuitableindexifpresent.
Revisionidhandling
Incompatiblechangesin3.0
522
Theexists()methodofacollectionnowthrowsanexceptionwhenthespecifieddocumentexistsbutitsrevisioniddoesnotmatchtherevisionidspecified.PreviousversionsofArangoDBsimplyreturnedfalseifeithernodocumentexistedwiththespecifiedkeyorwhentherevisioniddidnotmatch.Itwasthereforeimpossibletodistinguishthesetwocasesfromthereturnvaluealone.3.0correctsthis.Additionally,exists()inpreviousversionsalwaysreturnedabooleanifonlythedocumentkeywasgiven.3.0nowreturnsthedocument'smeta-data,whichincludesthedocument'scurrentrevisionid.
GiventhereisadocumentwithkeytestincollectionmyCollection,thenthebehaviorof3.0isasfollows:
/*testifdocumentexists.thisreturnedtruein2.8*/
db.myCollection.exists("test");
{
"_key":"test",
"_id":"myCollection/test",
"_rev":"9758059"
}
/*testifdocumentexists.thisreturnedtruein2.8*/
db.myCollection.exists({_key:"test"});
{
"_key":"test",
"_id":"myCollection/test",
"_rev":"9758059"
}
/*testifdocumentexists.thisalsoreturnedfalsein2.8*/
db.myCollection.exists("foo");
false
/*testifdocumentwithagivenrevisionidexists.thisreturnedtruein2.8*/
db.myCollection.exists({_key:"test",_rev:"9758059"});
{
"_key":"test",
"_id":"myCollection/test",
"_rev":"9758059"
}
/*testifdocumentwithagivenrevisionidexists.thisreturnedfalsein2.8*/
db.myCollection.exists({_key:"test",_rev:"1234"});
JavaScriptexception:ArangoError1200:conflict
Capconstraints
Thecapconstraintsfeaturehasbeenremoved.Thischangehasledtotheremovalofthecollectionoperationsfirst()andlast(),whichwereinternallybasedondatafromcapconstraints.
AscapconstraintshavebeenremovedinArangoDB3.0itisnotpossibletocreateanindexoftype"cap"withacollection'sensureIndex()function.ThededicatedfunctionensureCapConstraint()hasalsobeenremovedfromthecollectionAPI.
GraphBlueprintsJSModule
Thedeprecatedmodulegraph-blueprintshasbeendeleted.Allit'sfeaturesarecoveredbythegeneral-graphmodule.
GeneralGraphFluentAQLinterface
ThefluentinterfacehasbeenremovedfromArangoDB.It'sfeatureswerecompletelyoverlappingwith"aqb"whichcomespreinstalledaswell.PleaseswitchtoAQBinstead.
UndocumentedAPIs
TheundocumentedfunctionsBY_EXAMPLE_HASH()andBY_EXAMPLE_SKIPLIST(),BY_CONDITION_SKIPLIST,CPP_NEIGHBORSandCPP_SHORTEST_PATHhavebeenremoved.ThesefunctionswerealwayshiddenandnotintendedtobepartofthepublicJavaScriptAPIforcollections.
HTTPAPIchanges
Incompatiblechangesin3.0
523
CRUDoperations
ThefollowingincompatiblechangeshavebeenmadetotheHTTPAPIinArangoDB3.0:
General
TheHTTPinsertoperationsforsingledocumentsandedges(POST/_api/document)donotsupporttheURLparameter"createCollection"anymore.InpreviousversionsofArangoDBthisparametercouldbeusedtoautomaticallycreateacollectionuponinsertionofthefirstdocument.ItisnowrequiredthatthetargetcollectionalreadyexistswhenusingthisAPI,otherwiseitwillreturnanHTTP404error.ThesameistruefortheimportAPIatPOST/_api/import.
CollectionscanstillbecreatedeasilyviaaseparatecalltoPOST/_api/collectionasbefore.
The"location"HTTPheaderreturnedbyArangoDBwheninsertinganewdocumentoredgenowalwayscontainsthedatabasename.ThiswasalsothedefaultbehaviorinpreviousversionsofArangoDB,butitcouldbeoverriddenbyclientssendingtheHTTPheaderx-arango-version:1.4intherequest.ClientscancontinuetosendthisheadertoArangoDB3.0,buttheheaderwillnotinfluencethelocationresponseheadersproducedbyArangoDB3.0anymore.
AdditionallytheCRUDoperationsAPIsdonotreturnanattribute"error"intheresponsebodywithanattributevalueof"false"incaseanoperationsucceeded.
Revisionidhandling
Theoperationsforupdating,replacingandremovingdocumentscanoptionallychecktherevisionnumberofthedocumenttobeupdated,replacedorremovedsothecallercanensuretheoperationworksonaspecificversionofthedocumentandtherearenolostupdates.
PreviousversionsofArangoDBallowedpassingtherevisionidofthepreviousdocumenteitherintheHTTPheaderIf-MatchorintheURLparameterrev.Forexample,removingadocumentwithaspecificrevisionidcouldbeachievedasfollows:
curl-XDELETE\
"http://127.0.0.1:8529/_api/document/myCollection/myKey?rev=123"
ArangoDB3.0doesnotsupportpassingtherevisionidviathe"rev"URLparameteranymore.InsteadthepreviousrevisionidmustbepassedintheHTTPheaderIf-Match,e.g.
curl-XDELETE\
--header"If-Match:'123'"\
"http://127.0.0.1:8529/_api/document/myCollection/myKey"
TheURLparameter"policy"wasalsousableinpreviousversionsofArangoDBtocontrolrevisionhandling.Usingitwasredundanttospecifyingtheexpectedrevisionidviathe"rev"parameteror"If-Match"HTTPheaderandthereforesupportforthe"policy"parameterwasremovedin3.0.
Inordertocheckforapreviousrevisionidwhenupdating,replacingorremovingdocumentspleaseusetheIf-MatchHTTPheaderasdescribedabove.WhennorevisioncheckifrequiredtheHTTPheadercanbeomitted,andtheoperationswillworkonthecurrentrevisionofthedocument,regardlessofitsrevisionid.
AlldocumentsAPI
TheHTTPAPIforretrievingtheids,keysorURLsofalldocumentsfromacollectionwaspreviouslylocatedatGET/_api/document?collection=....ThisAPIwasmovedtoPUT/_api/simple/all-keysandisnowexecutedasanAQLquery.ThenameofthecollectionmustnowbepassedintheHTTPrequestbodyinsteadofintherequestURL.Thesameistrueforthe"type"parameter,whichcontrolsthetypeoftheresulttobecreated.
CallstothepreviousAPIcanbetranslatedasfollows:
old:GET/_api/document?collection=<collection>&type=<type>withoutHTTPrequestbody3.0:PUT/_api/simple/all-keyswithHTTPrequestbody{"collection":"<collection>","type":"id"}
TheresultformatofthisAPIhasalsochangedslightly.InpreviousversionscallstotheAPIreturnedaJSONobjectwithadocumentsattribute.AsthefunctionalityisbasedonAQLinternallyin3.0,theAPInowreturnsaJSONobjectwitharesultattribute.
Incompatiblechangesin3.0
524
EdgesAPI
CRUDoperations
TheAPIfordocumentsandedgeshavebeenunifiedinArangoDB3.0.TheCRUDoperationsfordocumentsandedgesarenowhandledbythesameendpointat/_api/document.ForCRUDoperationsthereisnodistinctionanymorebetweendocumentsandedgesAPI-wise.
ThatmeansCRUDoperationsconcerningedgesneedtobesenttotheHTTPendpoint/_api/documentinsteadof/_api/edge.Sendingrequeststo/_api/edgewillresultinanHTTP404errorin3.0.Thefollowingmethodsareavailableat/_api/documentfordocumentsandedge:
HTTPPOST:insertnewdocumentoredgeHTTPGET:fetchanexistingdocumentoredgeHTTPPUT:replaceanexistingdocumentoredgeHTTPPATCH:partiallyupdateanexistingdocumentoredgeHTTPDELETE:removeanexistingdocumentoredge
WhencompletelyreplacinganedgeviaHTTPPUTpleasenotethatthereplacingedgedatanowneedstocontainthe_fromand_toattributesfortheedge.PreviousversionsofArangoDBdidnotrequiresending_fromand_towhenreplacingedges,as_fromand_tovalueswereimmutableforexistingedges.
The_fromand_toattributesofedgesnowalsoneedtobepresentinsidetheedgesobjectssenttotheserver:
curl-XPOST\
--data'{"value":1,"_from":"myVertexCollection/1","_to":"myVertexCollection/2"}'\
"http://127.0.0.1:8529/_api/document?collection=myEdgeCollection"
PreviousversionsofArangoDBrequiredthe_fromand_toattributesofedgesbesentseparatelyinURLparameterfromandto:
curl-XPOST\
--data'{"value":1}'\
"http://127.0.0.1:8529/_api/edge?collection=e&from=myVertexCollection/1&to=myVertexCollection/2"
Queryingconnectededges
TheRESTAPIforqueryingconnectededgesatGET/_api/edges/<collection>willnowmaketheedgeidsuniquebeforereturningtheconnectededges.Thisisprobablydesiredanywayasresultswillnowbereturnedonlyonceperdistinctinputedgeid.However,itmaybreakclientapplicationsthatrelyontheoldbehavior.
GraphAPI
Somedata-modificationoperationsinthenamedgraphsAPIat/_api/gharialnowreturneitherHTTP202(Accepted)orHTTP201(Created)iftheoperationsucceeds.WhichstatuscodeisreturneddependsonthewaitForSyncattributeoftheaffectedcollection.InpreviousversionssomeoftheseoperationsreturnHTTP200regardlessofthewaitForSyncvalue.
ThedeprecatedgraphAPI/_api/graphhasbeenremoved.Allit'sfeaturescanbereplacedusing/_api/gharialandAQLinstead.
SimplequeriesAPI
TheRESTroutesPUT/_api/simple/firstand/_api/simple/lasthavebeenremovedentirely.TheseAPIswereresponsibleforreturningthefirst-insertedandlast-inserteddocumentsinacollection.Thisfeaturewasbuiltoncapconstraintsinternally,whichhavebeenremovedin3.0.
Callingoneoftheseendpointsin3.0willresultinanHTTP404error.
IndexesAPI
Incompatiblechangesin3.0
525
Itisnotsupportedin3.0tocreateanindexwithtypecap(capconstraint)in3.0asthecapconstraintsfeaturehasbeeremoved.CallingtheindexcreationendpointHTTPAPIPOST/_api/index?collection=...withanindextypecapwillthereforeresultinanHTTP400error.
LogentriesAPI
TheRESTrouteHTTPGET/_admin/logisnowaccessiblefromwithinalldatabases.InpreviousversionsofArangoDB,thisroutewasaccessiblefromwithinthe_systemdatabaseonly,andanHTTP403(Forbidden)wasthrownbytheserverforanyaccessfromwithinanotherdatabase.
FiguresAPI
TheRESTrouteHTTPGET/_api/collection/<collection>/figureswillnotreturnthefollowingresultattributesastheybecamemeaninglessin3.0:
shapefiles.countshapes.fileSizeshapes.countshapes.sizeattributes.countattributes.size
DatabasesandCollectionsAPIs
WhencreatingadatabaseviatheAPIPOST/_api/database,ArangoDBwillnowalwaysreturntheHTTPstatuscode202(created)iftheoperationsucceeds.PreviousversionsofArangoDBreturnedHTTP202aswell,butthisbehaviorwaschangablebysendinganHTTPheaderx-arango-version:1.4.Whensendingthisheader,previousversionsofArangoDBreturnedanHTTPstatuscode200(ok).ClientscanstillsendthisheadertoArangoDB3.0butthiswillnotinfluencetheHTTPstatuscodeproducedbyArangoDB.
The"location"headerproducedbyArangoDB3.0willnowalwayscontainthedatabasename.ThiswasalsothedefaultinpreviousversionsofArangoDB,butthebehaviorcouldbeoverriddenbysendingtheHTTPheaderx-arango-version:1.4.Clientscanstillsendtheheader,butthiswillnotmakethedatabasenameinthe"location"responseheaderdisappear.
TheresultformatforqueryingallcollectionsviatheAPIGET/_api/collectionhasbeenchanged.
PreviousversionsofArangoDBreturnedanobjectwithanattributenamedcollectionsandanattributenamednames.Bothcontainedallavailablecollections,butcollectionscontainedthecollectionsasanarray,andnamescontainedthecollectionsagain,containedinanobjectinwhichtheattributenameswerethecollectionnames,e.g.
{
"collections":[
{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
...
],
"names":{
"test":{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
"something":{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
...
}
}
Thisresultstructurewasredundant,andthereforehasbeensimplifiedtojust
{
"result":[
{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
...
]
}
inArangoDB3.0.
Incompatiblechangesin3.0
526
ReplicationAPIs
TheURLparameter"failOnUnknown"wasremovedfromtheRESTAPIGET/_api/replication/dump.Thisparametercontrolledwhetherdumpingorreplicatingedgesshouldfailifoneofthevertexcollectionslinkedintheedge's_fromor_toattributeswasnotpresentanymore.Inthiscasethe_fromand_tovaluescouldnotbetranslatedintomeaningfulidsanymore.
Thereweretwowaysforhandlingthis:
settingfailOnUnknowntotruecausedtheHTTPrequesttofail,leavingerrorhandlingtotheusersettingfailOnUnknowntofalsecausedtheHTTPrequesttocontinue,translatingthecollectionnamepartinthe_fromor_tovalueto_unknown.
InArangoDB3.0thisparameterisobsolete,as_fromand_toarestoredasself-containedstringvaluesallthetime,sotheycannotgetinvalidwhenreferencedcollectionsaredropped.
TheresultformatoftheAPIGET/_api/replication/logger-followhaschangedslightlyinthefollowingaspects:
documentsandedgesarereportedinthesameway.Thetypefordocumentinsertions/updatesandedgeinsertions/updatesisnowalways2300.PreviousversionsofArangoDBreturnedatypevalueof2300fordocumentsand2301foredges.recordsaboutinsertions,updatesorremovalsofdocumentsandedgesdonothavethekeyandrevattributesonthetop-levelanymore.Instead,keyandrevcanbeaccessedbypeekingintothe_keyand_revattributesofthedatasub-attributesofthechangerecord.
Thesameistrueforthecollection-specificchangesAPIGET/_api/replication/dump.
UsermanagementAPIs
TheRESTAPIendpointPOST/_api/userforaddingnewusersnowrequirestherequesttocontainaJSONobjectwithanattributenameduser,containingthenameoftheusertobecreated.PreviousversionsofArangoDBalsocheckedthisattribute,butadditionallylookedforanattributeusernameiftheuserattributedidnotexist.
UndocumentedAPIs
ThefollowingundocumentedHTTPRESTendpointshavebeenremovedfromArangoDB'sRESTAPI:
/_open/cerberusand/_system/cerberus:theseendpointswereintendedforsomeArangoDB-internalapplicationsonlyPUT/_api/simple/by-example-hash,PUT/_api/simple/by-example-skiplistandPUT/_api/simple/by-condition-skiplist:thesemethodsweredocumentedinearlyversionsofArangoDBbuthavebeenmarkedasnotintendedtobecalledbyenduserssinceArangoDBversion2.3.ThesemethodsshouldnothavebeenpartofanyArangoDBmanualsinceversion2.4./_api/structure:anolderunfinishedandunpromotedAPIfordataformatandtypechecks,supersededbyFoxxapplications.
AdministrationAPIs
/_admin/shutdownnowneedstobecalledwiththeHTTPDELETEmethod
HandlingofCORSrequests
ItcannowbecontrolledindetailforwhichoriginhostsCORS(Cross-originresourcesharing)requestswithcredentialswillbeallowed.ArangoDB3.0providesthestartupoption--http.trusted-originthatcanbeusedtospecifyoneormanyoriginsfromwhichCORSrequestsaretreatedas"trustworthy".
Theoptioncanbespecifiedmultipletimes,oncepertrustedorigin,e.g.
--http.trusted-originhttp://127.0.0.1:8529--http.trusted-originhttps://127.0.0.1:8599
ThiswillmaketheArangoDBserverrespondtoCORSrequestsfromtheseoriginswithanAccess-Control-Allow-CredentialsHTTPheaderwithavalueoftrue.WebbrowserscaninspectthisheaderandcanallowpassingArangoDBwebinterfacecredentials(ifstoredinthebrowser)totherequestingsite.ArangoDBwillnotforwardorprovideanycredentials.
Incompatiblechangesin3.0
527
SettingthisoptionisonlyrequiredifapplicationsonotherhostsneedtoaccesstheArangoDBwebinterfaceorotherHTTPRESTAPIsfromawebbrowserwiththesamecredentialsthattheuserhasenteredwhenloggingintothewebinterface.WhenawebbrowserfindstheAccess-Control-Allow-CredentialsHTTPresponseheader,itmayforwardthecredentialsenteredintothebrowserfortheArangoDBwebinterfacelogintotheothersite.
Thisisapotentialsecurityissue,sotherearenotrustedoriginsbydefault.Itmayberequiredtosetsometrustedoriginsifyou'replanningtoissueAJAXrequeststoArangoDBfromothersitesfromthebrowser,withthecredentialsenteredduringtheArangoDBinterfacelogin(i.e.singlesign-on).Ifsuchfunctionalityisnotused,theoptionshouldnotbeset.
Tospecifyatrustedorigin,specifytheoptiononcepertrustedoriginasshownabove.NotethatthetrustedoriginvaluesspecifiedinthisoptionwillbecomparedbytewisewiththeOriginHTTPheadervaluesentbyclients,andonlyexactmatcheswillpass.
ThereisalsothewildcardallforenablingCORSaccessfromalloriginsinatestordevelopmentsetup:
--http.trusted-originall
SettingthisoptionwillleadtotheArangoDBserverrespondingwithanAccess-Control-Allow-Credentials:trueHTTPheadertoallincomingCORSrequests.
Command-lineoptionsQuiteafewstartupoptionsinArangoDB2weredoublenegations(like--server.disable-authenticationfalse).InArangoDB3thesearenowexpressedaspositives(e.g.--server.authentication).AlsotheoptionsbetweentheArangoDBserveranditsclienttoolshavebeingunified.Forexample,theloggeroptionsarenowthesamefortheserverandtheclienttools.Additionallymanyoptionshavebeenmovedintomoreappropriatetopicsections.
Renamedoptions
Thefollowingoptionshavebeenavailablebefore3.0andhavechangedtheirnamein3.0:
--server.disable-authenticationwasrenamedto--server.authentication.Notethatthemeaningoftheoption--server.authenticationistheoppositeoftheprevious--server.disable-authentication.--server.disable-authentication-unix-socketswasrenamedto--server.authentication-unix-sockets.Notethatthemeaningoftheoption--server.authentication-unix-socketsistheoppositeoftheprevious--server.disable-authentication-unix-sockets.--server.authenticate-system-onlywasrenamedto--server.authentication-system-only.Themeaningoftheoptioninunchanged.--server.disable-statisticswasrenamedto--server.statistics.Notethatthemeaningoftheoption--server.statisticsistheoppositeoftheprevious--server.disable-statistics.--server.cafilewasrenamedto--ssl.cafile.Themeaningoftheoptionisunchanged.--server.keyfilewasrenamedto--ssl.keyfile.Themeaningoftheoptionisunchanged.--server.ssl-cachewasrenamedto--ssl.session-cache.Themeaningoftheoptionisunchanged.--server.ssl-cipher-listwasrenamedto--ssl.cipher-list.Themeaningoftheoptionisunchanged.--server.ssl-optionswasrenamedto--ssl.options.Themeaningoftheoptionisunchanged.--server.ssl-protocolwasrenamedto--ssl.protocol.Themeaningoftheoptionisunchanged.--server.backlog-sizewasrenamedto--tcp.backlog-size.Themeaningoftheoptionisunchanged.--server.reuse-addresswasrenamedto--tcp.reuse-address.Themeaningoftheoptionisunchanged.--server.disable-replication-applierwasrenamedto--database.replication-applier.Themeaningoftheoption--database.replication-applieristheoppositeoftheprevious--server.disable-replication-applier.--server.allow-method-overridewasrenamedto--http.allow-method-override.Themeaningoftheoptionisunchanged.--server.hide-product-headerwasrenamedto--http.hide-product-header.Themeaningoftheoptionisunchanged.--server.keep-alive-timeoutwasrenamedto--http.keep-alive-timeout.Themeaningoftheoptionisunchanged.--server.foxx-queueswasrenamedto--foxx.queues.Themeaningoftheoptionisunchanged.--server.foxx-queues-poll-intervalwasrenamedto--foxx.queues-poll-interval.Themeaningoftheoptionisunchanged.--no-serverwasrenamedto--server.rest-server.Notethatthemeaningoftheoption--server.rest-serveristheoppositeoftheprevious--no-server.--database.query-cache-modewasrenamedto--query.cache-mode.Themeaningoftheoptionisunchanged.--database.query-cache-max-resultswasrenamedto--query.cache-entries.Themeaningoftheoptionisunchanged.
Incompatiblechangesin3.0
528
--database.disable-query-trackingwasrenamedto--query.tracking.Themeaningoftheoption--query.trackingistheoppositeoftheprevious--database.disable-query-tracking.--log.ttywasrenamedto--log.foreground-tty.Themeaningoftheoptionisunchanged.--upgradehasbeenrenamedto--database.auto-upgrade.Incontrastto2.8thisoptionnowrequiresabooleanparameter.Toactuallyperformanautomaticdatabaseupgradeatstartupuse--database.auto-upgradetrue.Tonotperformit,use--database.auto-upgradefalse.--check-versionhasbeenrenamedto--database.check-version.--temp-pathhasbeenrenamedto--temp.path.
Logverbosity,topicsandoutputfiles
Loggingnowsupportslogtopics.Youcancontrolthesebyspecifyingalogtopicinfrontofalogleveloranoutput.Forexample
--log.levelstartup=trace--log.levelinfo
willlogmessagesconcerningstartupattracelevel,everythingelseatinfolevel.--log.levelcanbespecifiedmultipletimesatstartup,forasmanytopicsasneeded.
Somerelevantlogtopicsavailablein3.0are:
collector:informationabouttheWALcollector'sstatecompactor:informationaboutthecollectiondatafilecompactordatafiles:datafile-relatedoperationsmmap:informationaboutmemory-mappingoperationsperformance:someperformance-relatedinformationqueries:executedAQLqueriesreplication:replication-relatedinforequests:HTTPrequestsstartup:informationaboutserverstartupandshutdownthreads:informationaboutthreads
Thenewlogoption--log.output<definition>allowsdirectingtheglobalorper-topiclogoutputtodifferentoutputs.Theoutputdefinition""canbeoneof
"-"forstdin"+"forstderr"syslog://""syslog:///""file://"
Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputfordifferentlogtopics.Tosetupaper-topicoutputconfiguration,use--log.output<topic>=<definition>,e.g.
queries=file://queries.txt
logsallqueriestothefile"queries.txt".
Theoldoption--log.fileisstillavailablein3.0forconveniencereasons.In3.0itisashortcutforthemoregeneraloption--log.outputfile://filename.
Theoldoption--log.requests-fileisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption--log.outputrequests=file://....
Theoldoption--log.performanceisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption--log.levelperformance=trace.
Removedoptionsforlogging
Theoptions--log.content-filterand--log.source-filterhavebeenremoved.TheyhavemostbeenusedduringArangoDB'sinternaldevelopment.
Incompatiblechangesin3.0
529
Thesyslog-relatedoptions--log.applicationand--log.facilityhavebeenremoved.Theyaresupersededbythemoregeneral--log.outputoptionwhichcanalsohandlesyslogtargets.
Removedotheroptions
Theoption--server.default-api-compatibilitywaspresentinearlierversionofArangoDBtocontrolvariousaspectsoftheserverbehavior,e.g.HTTPreturncodesortheformatofHTTP"location"headers.ClientapplicationscouldsendanHTTPheader"x-arango-version"withaversionnumbertorequesttheserverbehaviorofacertainArangoDBversion.
Thisoptionwasonlyhonoredinahandfulofcases(describedabove)andwasremovedin3.0becausethechangesinserverbehaviorcontrolledbythisoptionwerechangedevenbeforeArangoDB2.0.Thisshouldhaveleftenoughtimeforclientapplicationstoadapttothenewbehavior,makingtheoptionsuperfluousin3.0.
Threadoptions
Theoptions--server.threadsand--scheduler.threadsnowhaveadefaultvalueof0.When--server.threadsissetto0onstartup,thesuitablenumberofthreadswillbedeterminedbyArangoDBbyaskingtheOSforthenumberofavailableCPUsandusingthatasabaseline.IfthenumberofCPUsislowerthan4,ArangoDBwillstillstart4dispatcherthreads.When--scheduler.threadsissetto0,thenArangoDBwillautomaticallydeterminethenumberofschedulerthreadstostart.Thiswillnormallycreate2schedulerthreads.
Iftheexactnumberofthreadsneedstobesetbytheadmin,thenitisstillpossibletoset--server.threadsand--scheduler.threadstonon-zerovalues.ArangoDBwillusethesevaluesandstartthatmanythreads(notethatsomethreadsmaybecreatedlazilysotheymaynotbepresentdirectlyafterstartup).
ThenumberofV8JavaScriptcontextstobecreated(--javascript.v8-contexts)nowhasadefaultvalueof0too,meaningthatArangoDBwillcreateasmanyV8contextsastherewillbedispatcherthreads(controlledbythe--server.threadsoption).Settingthisoptiontoanon-zerovaluewillcreateexactlyasmanyV8contextsasspecified.
Settingtheseoptionsexplicitlytonon-zerovaluesmaybebeneficialinenvironmentsthathavefewresources(processingtime,maximumthreadcount,availablememory).
AuthenticationThedefaultvaluefor--server.authenticationisnowtrueintheconfigurationfilesshippedwithArangoDB.Thismeanstheserverwillbestartedwithauthenticationenabledbydefault,requiringallclientconnectionstoprovideauthenticationdatawhenconnectingtoArangoDBAPIs.PreviousArangoDBversionsusedthesetting--server.disable-authenticationtrue,effectivelydisablingauthenticationbydefault.
Thedefaultvaluefor--server.authentication-system-onlyisnowtrueinArangoDB.ThatmeansthatFoxxapplicationsrunninginArangoDBwillbepublicaccessible(atleasttheywillnotuseArangoDB'sbuiltinauthenticationmechanism).OnlyrequeststoArangoDBAPIsatURLpathprefixes/_api/and/_adminwillrequireauthentication.Tochangethat,andusethebuiltinauthenticationmechanismforFoxxapplicationstoo,set--server.authentication-system-onlytofalse,andmakesuretohavetheoption--server.authenticationsettotrueaswell.
Thoughenablingtheauthenticationisrecommendedforproductionsetups,itmaybeoverkillinadevelopmentenvironment.Toturnoffauthentication,theoption--server.authenticationcanbesettofalseinArangoDB'sconfigurationfileoronthecommand-line.
WebAdminInterface
TheJavaScriptshellhasbeenremovedfromArangoDB'swebinterface.ThefunctionalitytheshellprovidedisstillfullyavailableintheArangoShell(arangosh)binaryshippedwithArangoDB.
ArangoShellandclienttools
Incompatiblechangesin3.0
530
TheArangoShell(arangosh)andtheotherclienttoolsbundledwithArangoDBcanonlyconnecttoanArangoDBserverofversion3.0orhigher.TheywillnotconnecttoanArangoDB2.8.ThisisbecausetheserverHTTPAPIshavechangedbetween2.8and3.0,andallclienttoolsusestheseAPIs.
InordertoconnecttoearlierversionsofArangoDBwiththeclienttools,anolderversionoftheclienttoolsneedstobekeptinstalled.
ThepreferrednameforthetemplatestringgeneratorfunctionaqlQueryisnowaqlandisautomaticallyavailableinarangosh.Elsewhere,itcanbeloadedlikeconstaql=require('@arangodb').aql.
Command-lineoptionsadded
Allclienttoolsin3.0provideanoption--server.max-packet-sizeforcontrollingthemaximumsizeofHTTPpacketstobehandledbytheclienttools.Thedefaultvalueis128MB,asinpreviousversionsofArangoDB.Incontrasttopreviousversionsinwhichthevaluewashard-coded,theoptionisnowconfigurable.ItcanbeincreasedtomaketheclienttoolshandleverylargeHTTPresultmessagessentbytheserver.
Command-lineoptionschanged
Forallclienttools,theoption--server.disable-authenticationwasrenamedto--server.authentication.Notethatthemeaningoftheoption--server.authenticationistheoppositeoftheprevious--server.disable-authentication.
Theoption--server.ssl-protocolwasrenamedto--ssl.protocol.Themeaningoftheoptionisunchanged.
Thecommand-lineoption--quietwasremovedfromallclienttoolsexceptarangoshbecauseithadnoeffectinthem.
Arangobench
Inordertomakeitspurposemoreapparenttheformerarangobclienttoolhasbeenrenamedtoarangobenchin3.0.
MiscellaneouschangesThechecksumcalculationalgorithmforthecollection.checksum()methodanditscorrespondingRESTAPIGET/_api/collection/<collection</checksumhaschangedin3.0.Checksumscalculatedin3.0willdifferfromchecksumscalculatedwith2.8orbefore.
TheArangoDBserverin3.0doesnotreadafileENDPOINTScontainingalistofadditionalendpointsonstartup.In2.8thisfilewasautomaticallyreadifpresentinthedatabasedirectory.
Thenamesofthesub-threadsstartedbyArangoDBhavechangedin3.0.ThisisrelevantonLinuxonly,wherethreadscanbenamedandthreadnamesmaybevisibletosystemtoolssuchastopormonitoringsolutions.
Incompatiblechangesin3.0
531
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.8.ArangoDB2.8alsocontainsseveralbugfixesthatarenotlistedhere.Foralistofbugfixes,pleaseconsulttheCHANGELOG.
AQLimprovements
AQLGraphTraversals/PatternMatching
AQLoffersanewfeaturetotraverseoveragraphwithoutwritingJavaScriptfunctionsbutwithalltheotherfeaturesyouknowfromAQL.Forthispurpose,aspecialversionofFORvariableNameINexpressionhasbeenintroduced.
Thisspecialversionhasthefollowingformat:FORvertex-variable,edge-variable,path-variableINtraversal-expression,wheretraversal-expressionhasthefollowingformat:[depth]directionstart-vertexgraph-definitionwiththefollowinginputparameters:
depth(optional):defineshowmanystepsareexecuted.Thevaluecaneitherbeanintegervalue(e.g.3)orarangeofintegervalues(e.g.1..5).Thedefaultis1.direction:defineswhichedgedirectionsarefollowed.CanbeeitherOUTBOUND,INBOUNDorANY.start-vertex:defineswherethetraversalisstarted.Mustbean_idvalueoradocument.graph-definition:defineswhichedgecollectionsareusedforthetraversal.MustbeeitherGRAPHgraph-nameforgraphscreatedwiththegraph-module,oralistofedgecollectionsedge-col1,edge-col2,..edge-colN.
Thethreeoutputvariableshavethefollowingsemantics:
vertex-variable:Thelastvisitedvertex.edge-variable:Thelastvisitededge(optional).path-variable:Thecompletepathfromstart-vertextovertex-variable(optional).
ThetraversalstatementcanbeusedinthesamewayastheoriginalFORvariableNameINexpression,andcanbecombinedwithfiltersandotherAQLconstructs.
AsanexampleonecannowfindthefriendsofafriendforacertainuserwiththisAQLstatement:
FORfoaf,e,pathIN2ANY@startUserGRAPH"relations"
FILTERpath.edges[0].type=="friend"
FILTERpath.edges[1].type=="friend"
FILTERfoaf._id!=@startUser
RETURNDISTINCTfoaf
Optimizerruleshavebeenimplementedtogainperformanceofthetraversalstatement.Theserulesmovefilterstatementsintothetraversalstatements.t.pathswhichcanneverpassthefilterarenotemittedtothevariables.
Asanexampletakethequeryaboveandassumethereareedgesthatdonothavetype=="friend".Ifinthefirstedgestepthereissuchanon-friendedgethesecondstepswillneverbecomputedfortheseedgesastheycannotfulfillthefiltercondition.
ArrayIndexes
Hashindexesandskiplistindexescannowoptionallybedefinedforarrayvaluessothattheyindexindividualarraymembersinsteadoftheentirearrayvalue.
Todefineanindexforarrayvalues,theattributenameisextendedwiththeexpansionoperator[*]intheindexdefinition.
Example:
db._create("posts");
db.posts.ensureHashIndex("tags[*]");
Whengiventhefollowingdocument
WhatsNewin2.8
532
{
"tags":[
"AQL",
"ArangoDB",
"Index"
]
}
thisindexwillnowcontaintheindividualvalues"AQL","ArangoDB"and"Index".
Nowtheindexcanbeusedforfindingalldocumentshaving"ArangoDB"somewhereintheirtagsarrayusingthefollowingAQLquery:
FORdocINposts
FILTER"ArangoDB"INdoc.tags[*]
RETURNdoc
Itisalsopossibletocreateanindexonsub-attributesofarrayvalues.Thismakessensewhentheindexattributeisanarrayofobjects,e.g.
db._drop("posts");
db._create("posts");
db.posts.ensureIndex({type:"hash",fields:["tags[*].name"]});
db.posts.insert({tags:[{name:"AQL"},{name:"ArangoDB"},{name:"Index"}]});
db.posts.insert({tags:[{name:"AQL"},{name:"2.8"}]});
Thefollowingquerywillthenusethearrayindex:
FORdocINposts
FILTER'AQL'INdoc.tags[*].name
RETURNdoc
Arrayvalueswillautomaticallybede-duplicatedbeforebeinginsertedintoanarrayindex.
PleasenotethatfilteringusingarrayindexesonlyworksfromwithinAQLqueriesandonlyifthequeryfiltersontheindexedattributeusingtheINoperator.Theothercomparisonoperators(==,!=,>,>=,<,<=)currentlydonotusearrayindexes.
Optimizerimprovements
TheAQLqueryoptimizercannowuseindexesifmultiplefilterconditionsonattributesofthesamecollectionarecombinedwithlogicalORs,andiftheusageofindexeswouldcompletelycovertheseconditions.
Forexample,thefollowingqueriescannowusetwoindependentindexesonvalue1andvalue2(thelatterqueryrequiresthattheindexesareskiplistindexesduetousageofthe<and>comparisonoperators):
FORdocINcollectionFILTERdoc.value1==42||doc.value2==23RETURNdoc
FORdocINcollectionFILTERdoc.value1<42||doc.value2>23RETURNdoc
Thenewoptimizerrule"sort-in-values"cannowpre-sorttheright-handsideoperandofINandNOTINoperatorssotheoperationcanuseabinarysearchwithlogarithmiccomplexityinsteadofalinearsearch.Therulewillbeappliedwhentheright-handsideoperandofanINorNOTINoperatorinafilterconditionisavariablethatisdefinedinadifferentloop/scopethantheoperatoritself.Additionally,thefilterconditionmustconsistofsolelytheINorNOTINoperationinordertoavoidanyside-effects.
Therulewillkickinforaqueriessuchasthefollowing:
LETvalues=/*someruntimeexpressionhere*/
FORdocINcollection
FILTERdoc.valueINvalues
RETURNdoc
Itwillnotbeappliedforthefollowigqueries,becausetheright-handsideoperandoftheINiseithernotavariable,orbecausetheFILTERconditionmayhavesideeffects:
WhatsNewin2.8
533
FORdocINcollection
FILTERdoc.valueIN/*someruntimeexpressionhere*/
RETURNdoc
LETvalues=/*someruntimeexpressionhere*/
FORdocINcollection
FILTERFUNCTION(doc.values)==23&&doc.valueINvalues
RETURNdoc
AQLfunctionsadded
ThefollowingAQLfunctionshavebeenaddedin2.8:
POW(base,exponent):returnsthebasetotheexponentexp
UNSET_RECURSIVE(document,attributename,...):recursivelyremovestheattributesattributename(canbeoneormany)fromdocumentanditssub-documents.Allotherattributeswillbepreserved.Multipleattributenamescanbespecifiedbyeitherpassingmultipleindividualstringargumentnames,orbypassinganarrayofattributenames:
UNSET_RECURSIVE(doc,'_id','_key','foo','bar')
UNSET_RECURSIVE(doc,['_id','_key','foo','bar'])
IS_DATESTRING(value):returnstrueifvalueisastringthatcanbeusedinadatefunction.Thisincludespartialdatessuchas2015or2015-10andstringscontaininginvaliddatessuchas2015-02-31.Thefunctionwillreturnfalseforallnon-stringvalues,evenifsomeofthemmaybeusableindatefunctions.
Miscellaneousimprovements
theArangoShellnowprovidestheconveniencefunctiondb._explain(query)forretrievingahuman-readableexplanationofAQLqueries.Thisfunctionisashorthandforrequire("org/arangodb/aql/explainer").explain(query).
theAQLqueryoptimizernowautomaticallyconvertsLENGTH(collection-name)toanoptimizedexpressionthatreturnsthenumberofdocumentsinacollection.PreviousversionsofArangoDBreturnedawarningwhenusingthisexpressionandalsoenumeratedalldocumentsinthecollection,whichwasinefficient.
improvedperformanceofskippingovermanydocumentsinanAQLquerywhennoindexesandnofiltersareused,e.g.
FORdocINcollection
LIMIT1000000,10
RETURNdoc
addedclusterexecutionsiteinfoinexecutionplanexplainoutputforAQLqueries
for30+AQLfunctionsthereisnowanadditionalimplementationinC++thatremovestheneedforinternaldataconversionwhenthefunctioniscalled
theAQLeditorinthewebinterfacenowsupportsusingbindparameters
Deadlockdetection
ArangoDB2.8nowhasanautomaticdeadlockdetectionfortransactions.
Adeadlockisasituationinwhichtwoormoreconcurrentoperations(usertransactionsorAQLqueries)trytoaccessthesameresources(collections,documents)andneedtowaitfortheotherstofinish,butnoneofthemcanmakeanyprogress.
Incaseofsuchadeadlock,therewouldbenoprogressforanyoftheinvolvedtransactions,andnoneoftheinvolvedtransactionscouldevercomplete.Thisiscompletelyundesirable,sothenewautomaticdeadlockdetectionmechanisminArangoDBwillautomaticallykickinandabortoneofthetransactionsinvolvedinsuchadeadlock.Abortingmeansthatallchangesdonebythetransactionwillberolledbackanderror29(deadlockdetected)willbethrown.
WhatsNewin2.8
534
Clientcode(AQLqueries,usertransactions)thataccessesmorethanonecollectionshouldbeawareofthepotentialofdeadlocksandshouldhandletheerror29(deadlockdetected)properly,eitherbypassingtheexceptiontothecallerorretryingtheoperation.
Replication
Thefollowingimprovementsforreplicationhavebeenmadein2.8(note:mostofthemhavebeenbackportedtoArangoDB2.7aswell):
addedautoResyncconfigurationparameterforcontinuousreplication.
Whensettotrue,areplicationslavewillautomaticallytriggerafulldatare-synchronizationwiththemasterwhenthemastercannotprovidethelogdatatheslavehadaskedfor.NotethatautoResyncwillonlyworkwhentheoptionrequireFromPresentisalsosettotrueforthecontinuousreplication,orwhenthecontinuoussyncerisstartedanddetectsthatnostarttickispresent.
Automaticre-synchronizationmaytransferalotofdatafromthemastertotheslaveandmaybeexpensive.Itisthereforeturnedoffbydefault.Whenturnedoff,theslavewillneverperformanautomaticre-synchronizationwiththemaster.
addedidleMinWaitTimeandidleMaxWaitTimeconfigurationparametersforcontinuousreplication.
Theseparameterscanbeusedtocontroltheminimumandmaximumwaittimetheslavewill(intentionally)idleandnotpollformasterlogchangesincasethemasterhadsentthefulllogsalready.TheidleMaxWaitTimevaluewillonlybeusedwhenadapativePollingissettotrue.WhenadaptivePollingisdisabled,onlyidleMinWaitTimewillbeusedasaconstanttimespaninwhichtheslavewillnotpollthemasterforfurtherchanges.Thedefaultvaluesare0.5secondsforidleMinWaitTimeand2.5secondsforidleMaxWaitTime,whichcorrespondtothehard-codedvaluesusedinpreviousversionsofArangoDB.
addedinitialSyncMaxWaitTimeconfigurationparameterforinitialandcontinuousreplication
Thisoptioncontrolsthemaximumwaittime(inseconds)thattheinitialsynchronizationwillwaitforaresponsefromthemasterwhenfetchinginitialcollectiondata.Ifnoresponseisreceivedwithinthistimeperiod,theinitialsynchronizationwillgiveupandfail.ThisoptionisalsorelevantforcontinuousreplicationincaseautoResyncissettotrue,asthenthecontinuousreplicationmaytriggerafulldatare-synchronizationincasethemastercannotthelogdatatheslavehadaskedfor.
HTTPrequestssentfromtheslavetothemasterduringinitialsynchronizationwillnowberetriediftheyfailwithconnectionproblems.
theinitialsynchronizationnowlogsitsprogresssoitcanbequeriedusingtheregularreplicationstatuscheckAPIs.
addedasyncattributeforsyncandsyncCollectionoperationscalledfromtheArangoShell.Setthingthisattributetotruewillmakethesynchronizationjobontheservergointothebackground,sothattheshelldoesnotblock.ThestatusofthestartedasynchronoussynchronizationjobcanbequeriedfromtheArangoShelllikethis:
/*startsinitialsynchronization*/
varreplication=require("org/arangodb/replication");
varid=replication.sync({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
async:true
});
/*nowquerytheidofthereturnedasyncjobandprintthestatus*/
print(replication.getSyncResult(id));
TheresultofgetSyncResult()willbefalsewhiletheserver-sidejobhasnotcompleted,anddifferenttofalseifithascompleted.Whenithascompleted,alljobresultdetailswillbereturnedbythecalltogetSyncResult().
thewebadmininterfacedashboardnowshowsaserver'sreplicationstatusatthebottomofthepage
WebAdminInterface
Thefollowingimprovementshavebeenmadeforthewebadmininterface:
theAQLeditornowhassupportforbindparameters.Thebindparametervaluescanbeeditedinthewebinterfaceandsavedwithaqueryforfutureuse.
WhatsNewin2.8
535
theAQLeditornowallowscancelingrunningqueries.Thiscanbeusedtocancellong-runningquerieswithoutswitchingtothequerymanagementsection.
thedashboardnowprovidesinformationabouttheserver'sreplicationstatusatthebottomofthepage.Thiscanbeusedtotrackeitherthestatusofaone-timesynchronizationorthecontinuousreplication.
thecompactionstatusandsomestatusinternalsaboutcollectionsarenowdisplayedinthedetailviewforacollectioninthewebinterface.Thesedatacanbeusedfordebuggingcompactionissues.
unloadingacollectionviathewebinterfacewillnowtriggergarbagecollectioninallv8contextsandforceaWALflush.Thisincreasesthechancesofperfomingtheunloadfaster.
thestatusterminologyforcollectionsforwhichanunloadrequesthasbeenissuedviathewebinterfacewaschangedfromintheprocessofbeingunloadedtowillbeunloaded.Thisismoreaccurateastheactualunloadmaybepostponeduntillateriftherearestillreferencespointingtodatainthecollection.
Foxximprovementsthemoduleresolutionusedbyrequirenowbehavesmorelikeinnode.js
theorg/arangodb/requestmodulenowreturnsresponsebodiesforerrorresponsesbydefault.Theoldbehaviorofnotreturningbodiesforerrorresponsescanbere-enabledbyexplicitlysettingtheoptionreturnBodyOnErrortofalse
Miscellaneouschanges
Thestartupoption--server.hide-product-headercanbeusedtomaketheservernotsendtheHTTPresponseheader"Server:ArangoDB"initsHTTPresponses.ThiscanbeusedtoconcealtheservermakefromHTTPclients.Bydefault,theoptionisturnedoffsotheheaderisstillsentasusual.
arangodumpandarangorestorenowhavebettererrorreporting.Additionally,arangodumpwillnowfailbydefaultwhentryingtodumpedgesthatrefertoalreadydroppedcollections.Thiscanbecircumventedbyspecifyingtheoption--forcetruewheninvokingarangodump.
arangoimpnowprovidesanoption--create-collection-typetospecifythetypeofthecollectiontobecreatedwhen--create-collectionissettotrue.Previously--create-collectionalwayscreateddocumentcollectionsandthecreationofedgecollectionswasnotpossible.
WhatsNewin2.8
536
IncompatiblechangesinArangoDB2.8ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB2.8,andadjustanyclientprogramsifnecessary.
AQL
Keywordsadded
ThefollowingAQLkeywordswereaddedinArangoDB2.8:
GRAPH
OUTBOUND
INBOUND
ANY
ALL
NONE
AGGREGATE
Usageofthesekeywordsforcollectionnames,variablenamesorattributenamesinAQLquerieswillnotbepossiblewithoutquoting.Forexample,thefollowingAQLquerywillstillworkasitusesaquotedcollectionnameandaquotedattributename:
FORdocIN`OUTBOUND`
RETURNdoc.`any`
Changedbehavior
TheAQLfunctionsNEARandWITHINnowhavestrictervalidationsfortheirinputparameterslimit,radiusanddistance.Theymaynowthrowexceptionswheninvalidparametersarepassedthatmayhavenotledtoexceptionsinpreviousversions.
Additionally,theexpansion([*])operatorinAQLhaschangeditsbehaviorwhenhandlingnon-arrayvalues:
InArangoDB2.8,callingtheexpansionoperatoronanon-arrayvaluewillalwaysreturnanemptyarray.PreviousversionsofArangoDBexpandednon-arrayvaluesbycallingtheTO_ARRAY()functionforthevalue,whichforexamplereturnedanarraywithasinglevalueforboolean,numericandstringinputvalues,andanarraywiththeobject'svaluesforanobjectinputvalue.Thisbehaviorwasinconsistentwithhowtheexpansionoperatorworksforthearrayindexesin2.8,sothebehaviorisnowunified:
iftheleft-handsideoperandof[*]isanarray,thearraywillbereturnedasiswhencalling[*]onitiftheleft-handsideoperandof[*]isnotanarray,anemptyarraywillbereturnedby[*]
AQLqueriesthatrelyontheoldbehaviorcanbechangedbyeithercallingTO_ARRAYexplicitlyorbyusingthe[*]atthecorrectposition.
Thefollowingexamplequerywillchangeitsresultin2.8comparedto2.7:
LETvalues="foo"RETURNvalues[*]
In2.7thequeryhasreturnedthearray["foo"],butin2.8itwillreturnanemptyarray[].Tomakeitreturnthearray["foo"]again,anexplicitTO_ARRAYfunctioncallisneededin2.8(whichinthiscaseallowstheremovalofthe[*]operatoraltogether).Thisalsoworksin2.7:
LETvalues="foo"RETURNTO_ARRAY(values)
Anotherexample:
LETvalues=[{name:"foo"},{name:"bar"}]
RETURNvalues[*].name[*]
Incompatiblechangesin2.8
537
Theabovereturned[["foo"],["bar"]]in2.7.In2.8itwillreturn[[],[]],becausethevalueofname`isnotanarray.Tochangetheresultstothe2.7style,thequerycanbechangedto
LETvalues=[{name:"foo"},{name:"bar"}]
RETURNvalues[*RETURNTO_ARRAY(CURRENT.name)]
Theabovealsoworksin2.7.Thefollowingtypesofquerieswon'tchange:
LETvalues=[1,2,3]RETURNvalues[*]
LETvalues=[{name:"foo"},{name:"bar"}]RETURNvalues[*].name
LETvalues=[{names:["foo","bar"]},{names:["baz"]}]RETURNvalues[*].names[*]
LETvalues=[{names:["foo","bar"]},{names:["baz"]}]RETURNvalues[*].names[**]
Deadlockhandling
Clientapplicationsshouldbepreparedtohandleerror29(deadlockdetected)thatArangoDBmaynowthrowwhenitdetectsadeadlockacrossmultipletransactions.Whenaclientapplicationreceiveserror29,itshouldretrytheoperationthatfailed.
TheerrorcanonlyoccurforAQLqueriesorusertransactionsthatinvolvemorethanasinglecollection.
Optimizer
TheAQLexecutionnodetypeIndexRangeNodewasreplacedwithanewmorecapableexecutionnodetypeIndexNode.ThatmeansinexecutionplanexplainoutputtherewillbenomoreIndexRangeNodesbutonlyIndexNode.Thisaffectsexplainoutputthatcanberetrievedviarequire("org/arangodb/aql/explainer").explain(query),db._explain(query),andtheHTTPqueryexplainAPI.
TheoptimizerrulethatmakesAQLqueriesactuallyuseindexeswasalsorenamedfromuse-index-rangetouse-indexes.Againthisaffectsexplainoutputthatcanberetrievedviarequire("org/arangodb/aql/explainer").explain(query),db._explain(query),andtheHTTPqueryexplainAPI.
Thequeryoptimizerruleremove-collect-intowasrenamedtoremove-collect-variables.Thisaffectsexplainoutputthatcanberetrievedviarequire("org/arangodb/aql/explainer").explain(query),db._explain(query),andtheHTTPqueryexplainAPI.
HTTPAPI
Whenaserver-sideoperationgotcanceledduetoanexplicitclientcancelrequestviaHTTPDELETE/_api/job,previousversionsofArangoDBreturnedanHTTPstatuscodeof408(requesttimeout)fortheresponseofthecanceledoperation.
TheHTTPreturncode408hascausedproblemswithsomeclientapplications.Somebrowsers(e.g.Chrome)handleda408responsebyresendingtheoriginalrequest,whichistheoppositeofwhatisdesiredwhenajobshouldbecanceled.
ThereforeArangoDBwillreturnHTTPstatuscode410(gone)forcanceledoperationsfromversion2.8on.
Foxx
ModelandRepository
DuetocompatibilityissuestheModelandRepositorytypesarenolongerimplementedasES2015classes.
Thepre-2.7"extend"stylesubclassingissupportedagainandwillnotemitanydeprecationwarnings.
varFoxx=require('org/arangodb/foxx');
varMyModel=Foxx.Model.extend({
//...
schema:{/*...*/}
});
Moduleresolution
Incompatiblechangesin2.8
538
ThebehavioroftheJavaScriptmoduleresolutionusedbytherequirefunctionhasbeenmodifiedtoimprovecompatibilitywithmoduleswrittenforNode.js.
Specifically
absolutepaths(e.g./some/absolute/path)arenowalwaysinterpretedasabsolutefilesystempaths,relativetothefilesystemroot
globalnames(e.g.global/name)arenowfirstintepretedasreferencestomodulesresidinginarelevantnode_modulesfolder,abuilt-inmoduleoramatchingdocumentintheinternal_modulescollection,andonlyresolvedtolocalfilepathsifnoothermatchisfound
Previouslythetwoformatsweretreatedinterchangeablyandwouldberesolvedtolocalfilepathsfirst,leadingtoproblemswhenlocalfilesusedthesamenamesasothermodules(e.g.alocalfilechai.jswouldcauseproblemswhentryingtoloadthechaimoduleinstalledinnode_modules).
Formoreinformationseetheblogannouncementofthischangeandtheupgradeguide.
Moduleorg/arangodb/request
Themodulenowalwaysreturnsresponsebodies,evenforerrorresponses.Inversionspriorto2.8themodulewouldsilentlydropresponsebodiesiftheresponseheaderindicatedanerror.
TheoldbehaviorofnotreturningbodiesforerrorresponsescanberestoredbyexplicitlysettingtheoptionreturnBodyOnErrortofalse:
letresponse=request({
//...
returnBodyOnError:false
});
Garbagecollection
TheV8garbagecollectionstrategywasslightlyadjustedsothatiteventuallyhappensinallV8contextsthatholdV8externalobjects(referencestoArangoDBdocumentsandcollections).Thisenablesabettercleanupoftheseresourcesandpreventsotherprocessessuchascompactionbeingstalledwhilewaitingfortheseresourcestobereleased.
InthiscontextthedefaultvaluefortheJavaScriptgarbagecollectionfrequency(--javascript.gc-frequency)wasalsoincreasedfrom10secondsto15seconds,aslessinternaloperationsinArangoDBarecarriedoutinJavaScript.
Clienttools
arangodumpwillnowfailbydefaultwhentryingtodumpedgesthatrefertoalreadydroppedcollections.Thiscanbecircumventedbyspecifyingtheoption--forcetruewheninvokingarangodump
Incompatiblechangesin2.8
539
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.7.ArangoDB2.7alsocontainsseveralbugfixesthatarenotlistedhere.Foralistofbugfixes,pleaseconsulttheCHANGELOG.
Performanceimprovements
Indexbuckets
Theprimaryindexesandhashindexesofcollectionscannowbesplitintomultipleindexbuckets.ThisoptionwasavailableforedgeindexesonlyinArangoDB2.6.
Abucketcanbeconsideredacontainerforaspecificrangeofindexvalues.Forprimary,hashandedgeindexes,determiningtheresponsiblebucketforanindexvalueisdonebyhashingtheactualindexvalueandapplyingasimplearithmeticoperationonthehash.
Becauseanindexvaluewillbepresentinatmostonebucketandbucketsareindependent,usingmultiplebucketsprovidesthefollowingbenefits:
initiallybuildingthein-memoryindexdatacanbeparallelizedevenforasingleindex,withonethreadperbucket(orwiththreadsbeingresponsibleformorethanonebucketatatime).Thiscanhelpreducingtheloadingtimeforcollections.
resizinganindexwhenitisabouttorunoutofreservespaceisperformedperbucket.Aseachbucketonlycontainsafractionoftheentireindex,resizingandrehashingabucketismuchfasterandlessintrusivethanresizingandrehashingtheentireindex.
Whencreatingnewcollections,thedefaultnumberofindexbucketsis8sinceArangoDB2.7.Inpreviousversions,thedefaultvaluewas1.Thenumberofbucketscanalsobeadjustedforexistingcollectionssotheycanbenefitfromtheoptimizations.Thenumberofindexbucketscanbesetforacollectionatanytimebyusingacollection'spropertiesfunction:
db.collection.properties({indexBuckets:16});
Thenumberofindexbucketsmustbeapowerof2.
Pleasenotethatforbuildingtheindexdataformultiplebucketsinparallelitisrequiredthatacollectioncontainsasignificantamountofdocumentsbecauseforalownumberofdocumentstheoverheadofparallelizationwilloutweighitsbenefits.Thecurrentthresholdvalueis256kdocuments,butthisvaluemaychangeinfutureversionsofArangoDB.Additionally,theconfigurationoption--database.index-threadswilldeterminehowmanyparallelthreadsmaybeusedforbuildingtheindexdata.
Fasterupdateandremoveoperationsinnon-uniquehashindexes
TheuniquehashindexesinArangoDBprovidedanamortizedO(1)lookup,insert,updateandremoveperformance.Non-uniquehashindexesprovidedamortizedO(1)insertperformance,buthadworseperformanceforupdateandremoveoperationsfornon-uniquevalues.Fordocumentswiththesameindexvalue,theymaintainedalistofcollisions.Whenadocumentwasupdatedorremoved,thatexactdocumenthadtobefoundinthecollisionslistfortheindexvalue.WhilegettingtothestartofthecollisionslistwasO(1),scanningthelisthadO(n)performanceintheworstcase(withnbeingthenumberofdocumentswiththesameindexvalue).Overall,thismadeupdateandremoveoperationsinnon-uniquehashindexesslowiftheindexcontainedmanyduplicatevalues.
ThishasbeenchangedinArangoDB2.7sothatnon-uniquehashindexesnowalsoprovideupdateandremoveoperationswithanamortizedcomplexityofO(1),eveniftherearemanyduplicates.
Resizingnon-uniquehashindexesnowalsodoesn'trequirelookingintothedocumentdata(whichmayinvolveadiskaccess)becausetheindexmaintainssomeinternalcachevalueperdocument.Whenresizingandrehashingtheindex(oranindexbucket),theindexwillfirstcompareonlythecachevaluesbeforepeekingintotheactualdocuments.Thischangecanalsoleadtoreducedindexresizingtimes.
Throughputenhancements
TheArangoDB-internalimplementationsfordispatchingrequests,keepingstatisticsandassigningV8contextstothreadshavebeenimprovedinordertouselesslocks.Thesechangesallowhigherconcurrencyandthroughputinthesecomponents,whichcanalsomaketheserverhandlemorerequestsinagivenperiodoftime.
WhatsNewin2.7
540
Whatgainscanbeexpecteddependsonwhichoperationsareexecuted,buttherearereal-worldcasesinwhichthroughputincreasedbybetween25%and70%whencomparedto2.6.
Madvisehints
TheLinuxvariantforArangoDBprovidestheOSwithmadvisehintsaboutindexmemoryanddatafilememory.Thesehintscanspeedupthingswhenmemoryistight,inparticularatcollectionloadtimebutalsoforrandomaccesseslater.ThereisnoformalguaranteethattheOSactuallyusesthemadvisehintsprovidedbyArangoDB,butactualmeasurementshaveshownimprovementsforloadingbiggercollections.
AQLimprovements
Additionaldatefunctions
ArangoDB2.7providesseveralextraAQLfunctionsfordateandtimecalculationandmanipulation.ThesefunctionswerecontributedbyGitHubusers@[email protected]!
Thefollowingextradatefunctionsareavailablefrom2.7on:
DATE_DAYOFYEAR(date):Returnsthedayofyearnumberofdate.Thereturnvaluesrangefrom1to365,or366inaleapyearrespectively.
DATE_ISOWEEK(date):ReturnstheISOweekdateofdate.Thereturnvaluesrangefrom1to53.Mondayisconsideredthefirstdayoftheweek.Therearenofractionalweeks,thusthelastdaysinDecembermaybelongtothefirstweekofthenextyear,andthefirstdaysinJanuarymaybepartofthepreviousyear'slastweek.
DATE_LEAPYEAR(date):Returnswhethertheyearofdateisaleapyear.
DATE_QUARTER(date):Returnsthequarterofthegivendate(1-based):
1:January,February,March2:April,May,June3:July,August,September4:October,November,December
DATE_DAYS_IN_MONTH(date):Returnsthenumberofdaysindate'smonth(28..31).
DATE_ADD(date,amount,unit):Addsamountgiveninunittodateandreturnsthecalculateddate.
unitcanbeeitherofthefollowingtospecifythetimeunittoaddorsubtract(case-insensitive):
y,year,yearsm,month,monthsw,week,weeksd,day,daysh,hour,hoursi,minute,minutess,second,secondsf,millisecond,milliseconds
amountisthenumberofunitstoadd(positivevalue)orsubtract(negativevalue).
DATE_SUBTRACT(date,amount,unit):Subtractsamountgiveninunitfromdateandreturnsthecalculateddate.
ItworksthesameasDATE_ADD(),exceptthatitsubtracts.ItisequivalenttocallingDATE_ADD()withanegativeamount,exceptthatDATE_SUBTRACT()canalsosubtractISOdurations.NotethatnegativeISOdurationsarenotsupported(i.e.startingwith-P,like-P1Y).
DATE_DIFF(date1,date2,unit,asFloat):Calculatethedifferencebetweentwodatesingiventimeunit,optionallywithdecimalplaces.Returnsanegativevalueifdate1isgreaterthandate2.
DATE_COMPARE(date1,date2,unitRangeStart,unitRangeEnd):Comparetwopartialdatesandreturntrueiftheymatch,falseotherwise.Thepartstocomparearedefinedbyarangeoftimeunits.
WhatsNewin2.7
541
Thefullrangeis:years,months,days,hours,minutes,seconds,milliseconds.PasstheunittostartfromasunitRangeStart,andtheunittoendwithasunitRangeEnd.Allunitsinbetweenwillbecompared.LeaveoutunitRangeEndtoonlycompareunitRangeStart.
DATE_FORMAT(date,format):Formatadateaccordingtothegivenformatstring.Itsupportsthefollowingplaceholders(case-insensitive):
%t:timestamp,inmillisecondssincemidnight1970-01-01%z:ISOdate(0000-00-00T00:00:00.000Z)%w:dayofweek(0..6)%y:year(0..9999)%yy:year(00..99),abbreviated(lasttwodigits)%yyyy:year(0000..9999),paddedtolengthof4%yyyyyy:year(-009999..+009999),withsignprefixandpaddedtolengthof6%m:month(1..12)%mm:month(01..12),paddedtolengthof2%d:day(1..31)%dd:day(01..31),paddedtolengthof2%h:hour(0..23)%hh:hour(00..23),paddedtolengthof2%i:minute(0..59)%ii:minute(00..59),paddedtolengthof2%s:second(0..59)%ss:second(00..59),paddedtolengthof2%f:millisecond(0..999)%fff:millisecond(000..999),paddedtolengthof3%x:dayofyear(1..366)%xxx:dayofyear(001..366),paddedtolengthof3%k:ISOweekdate(1..53)%kk:ISOweekdate(01..53),paddedtolengthof2%l:leapyear(0or1)%q:quarter(1..4)%a:daysinmonth(28..31)%mmm:abbreviatedEnglishnameofmonth(Jan..Dec)%mmmm:Englishnameofmonth(January..December)%www:abbreviatedEnglishnameofweekday(Sun..Sat)%wwww:Englishnameofweekday(Sunday..Saturday)%&:specialescapesequenceforrareoccasions%%:literal%%:ignored
RETURNDISTINCT
Toreturnuniquevaluesfromaquery,AQLnowprovidestheDISTINCTkeyword.ItcanbeusedasamodifierforRETURNstatements,asashorteralternativetothealreadyexistingCOLLECTstatement.
Forexample,thefollowingqueryonlyreturnsdistinct(unique)statusattributevaluesfromthecollection:
FORdocINcollection
RETURNDISTINCTdoc.status
RETURNDISTINCTisnotallowedonthetop-levelofaqueryifthereisnoFORloopinfrontofit.RETURNDISTINCTisallowedinsubqueries.
RETURNDISTINCTensuresthatthevaluesreturnedaredistinct(unique),butdoesnotguaranteeanyorderofresults.Inordertohavecertainresultorder,anadditionalSORTstatementmustbeaddedtoaquery.
Shorthandobjectnotation
WhatsNewin2.7
542
AQLnowprovidesashorthandnotationforobjectliteralsinthestyleofES6objectliterals:
LETname="Peter"
LETage=42
RETURN{name,age}
Thisisequivalenttothepreviouslyavailablecanonicalform,whichisstillavailableandsupported:
LETname="Peter"
LETage=42
RETURN{name:name,age:age}
Arrayexpansionimprovements
Thealreadyexisting[*]operatorhasbeenimprovedwithoptionalfilteringandprojectionandlimitcapabilities.
Forexample,considerthefollowingexamplequerythatfiltersvaluesfromanarrayattribute:
FORuINusers
RETURN{
name:u.name,
friends:(
FORfINu.friends
FILTERf.age>u.age
RETURNf.name
)
}
Withthe[*]operator,thisquerycanbesimplifiedto
FORuINusers
RETURN{name:u.name,friends:u.friends[*FILTERCURRENT.age>u.age].name}
Thepseudo-variableCURRENTcanbeusedtoaccessthecurrentarrayelement.TheFILTERconditioncanrefertoCURRENToranyvariablesvalidintheouterscope.
Toreturnaprojectionofthecurrentelement,therecannowbeaninlineRETURN:
FORuINusers
RETURNu.friends[*RETURNCONCAT(CURRENT.name,"isafriendof",u.name)]
whichisthesimplifiedvariantfor:
FORuINusers
RETURN(
FORfriendINu.friends
RETURNCONCAT(friend.name,"isafriendof",u.name)
)
Arraycontraction
Inordertocollapse(orflatten)resultsinnestedarrays,AQLnowprovidesthe[**]operator.Itworkssimilartothe[*]operator,butadditionallycollapsesnestedarrays.Howmanylevelsarecollapsedisdeterminedbytheamountof*charactersused.
Forexample,considerthefollowingquerythatproducesanestedresult:
FORuINusers
RETURNu.friends[*].name
The[**]operatorcannowbeappliedtogetridofthenestedarrayandturnitintoaflatarray.Wesimplyapplythe[**]onthepreviousqueryresult:
WhatsNewin2.7
543
RETURN(
FORuINusersRETURNu.friends[*].name
)[**]
Templatequerystrings
AssemblingquerystringsinJavaScripthasbeenerror-pronewhenusingsimplestringconcatenation,especiallybecauseplainJavaScriptstringsdonothavemultiline-support,andbecauseofpotentialparameterinjectionissues.WhilemultilinequerystringscanbeassembledwithES6templatestringssinceArangoDB2.5,andquerybindparametersaretheresinceArangoDB1.0topreventparameterinjection,therewasnoJavaScript-ysolutiontocombinethese.
ArangoDB2.7nowprovidesanES6templatestringgeneratorfunctionthatcanbeusedtoeasilyandsafelyassembleAQLqueriesfromJavaScript.JavaScriptvariablesandexpressionscanbeusedeasilyusingregularES6templatestringsubstitutions:
letname='test';
letattributeName='_key';
letquery=aqlQuery`FORuINusers
FILTERu.name==${name}
RETURNu.${attributeName}`;
db._query(query);
ThisismorelegiblethanwhenusingaplainJavaScriptstringandalsodoesnotrequiredefiningthebindparametervaluesseparately:
letname='test';
letattributeName='_key';
letquery="FORuINusers"+
"FILTERu.name==@name"+
"RETURNu.@attributeName";
db._query(query,{
name,
attributeName
});
TheaqlQuerytemplatestringgeneratorwillalsohandlecollectionobjectsautomatically:
db._query(aqlQuery`FORuIN${db.users}RETURNu.name`);
NotethatwhiletemplatestringsareavailableintheJavaScriptfunctionsprovidedtobuildqueries,theyaren'tafeatureofAQLitself.AQLcouldalwayshandlemultilinequerystringsandprovidedbindparameters(@...)forseparatingthequerystringandtheparametervalues.TheaqlQuerytemplatestringgeneratorfunctionwilltakecareofthisseparation,too,butwilldoitbehindthescenes.
AQLqueryresultcache
TheAQLqueryresultcachecanoptionallycachethecompleteresultsofallorjustselectedAQLqueries.Itcanbeoperatedinthefollowingmodes:
off:thecacheisdisabled.Noqueryresultswillbestoredon:thecachewillstoretheresultsofallAQLqueriesunlesstheircacheattributeflagissettofalsedemand:thecachewillstoretheresultsofAQLqueriesthathavetheircacheattributesettotrue,butwillignoreallothers
Themodecanbesetatserverstartupusingthe--database.query-cache-modeconfigurationoptionandlaterchangedatruntime.Thedefaultvalueisoff,meaningthatthequeryresultcacheisdisabled.Thisisbecausethecachemayconsumeadditionalmemorytokeepqueryresults,andalsobecauseitmustbeinvalidatedwhenchangeshappenincollectionsforwhichresultshavebeencached.
Thequeryresultcachemaythereforehavepositiveornegativeeffectsonqueryexecutiontimes,dependingontheworkload:itwillnotmakemuchsenseturningonthecacheinwrite-onlyorwrite-mostlyscenarios,butthecachemaybeverybeneficialincaseworkloadsareread-onlyorread-mostly,andqueryarecomplex.
Ifthequerycacheisoperatedindemandmode,itcanbecontrolledperqueryifthecacheshouldbecheckedforaresult.
WhatsNewin2.7
544
Miscellaneouschanges
Optimizer
TheAQLoptimizerrulepatch-update-statementshasbeenadded.ThisrulecanoptimizecertainAQLUPDATEqueriesthatupdatedocumentsintheacollectionthattheyalsoiterateover.
Forexample,thefollowingqueryreadsdocumentsfromacollectioninordertoupdatethem:
FORdocINcollection
UPDATEdocWITH{newValue:doc.oldValue+1}INcollection
Inthiscase,onlyasinglecollectionisaffectedbythequery,andthereisnoindexlookupinvolvedtofindtheto-be-updateddocuments.Inthiscase,theUPDATEquerydoesnotrequiretakingafull,memory-intensivesnapshotofthecollection,butitcanbeperformedinsmallchunks.Thiscanleadtomemorysavingswhenexecutingsuchqueries.
Functioncallargumentsoptimization
ThisoptimizationwillleadtoargumentsinfunctioncallsinsideAQLqueriesnotbeingcopiedbutbeingpassedbyreference.ThismayspeedupcallstofunctionswithbiggerargumentvaluesorqueriesthatcallAQLfunctionsalotoftimes.
WebAdminInterface
Thewebinterfacenowhasanewdesign.
The"Applications"tabinthewebinterfaceshasbeenrenamedto"Services".
TheArangoDBAPIdocumentationhasbeenmovedfromthe"Tools"menutothe"Links"menu.ThenewdocumentationisbasedonSwagger2.0andopensinaseparatewebpage.
Foxximprovements
ES2015Classes
AllFoxxconstructorshavebeenreplacedwithES2015classesandcanbeextendedusingtheclasssyntax.TheextendmethodisstillsupportedatthemomentbutwillbecomedeprecatedinArangoDB2.8andremovedinArangoDB2.9.
Before:
varFoxx=require('org/arangodb/foxx');
varMyModel=Foxx.Model.extend({
//...
schema:{/*...*/}
});
After:
varFoxx=require('org/arangodb/foxx');
classMyModelextendsFoxx.Model{
//...
}
MyModel.prototype.schema={/*...*/};
Confidentialconfiguration
Itisnowpossibletospecifyconfigurationoptionswiththetypepassword.ThepasswordtypeisequivalenttothetexttypebutwillbemaskedinthewebfrontendtopreventaccidentalexposureofconfidentialoptionslikeAPIkeysandpasswordswhenconfiguringyourFoxxapplication.
WhatsNewin2.7
545
Dependencies
Thesyntaxforspecifyingdependenciesinmanifestshasbeenextendedtoallowspecifyingoptionaldependencies.Unmetoptionaldependencieswillnotpreventanappfrombeingmounted.Thetraditionalshorthandsyntaxforspecifyingnon-optionaldependencieswillstillbesupportedintheupcomingversionsofArangoDB.
Before:
{
...
"dependencies":{
"notReallyNeeded":"users:^1.0.0",
"totallyNecessary":"sessions:^1.0.0"
}
}
After:
{
"dependencies":{
"notReallyNeeded":{
"name":"users",
"version":"^1.0.0",
"required":false
},
"totallyNecessary":{
"name":"sessions",
"version":"^1.0.0"
}
}
}
Replication
TheexistingreplicationHTTPAPIhasbeenextendedwithmethodsthatreplicationclientscanusetodeterminewhetheragivendate,identifiedbyatickvalue,isstillpresentonamasterforreplication.BycallingtheseAPIs,clientscanmakeaninformeddecisionaboutwhetherthemastercanstillprovideallmissingdatastartingfromthepointuptowhichtheclienthadalreadysynchronized.Thiscanbehelpfulincaseareplicationclientisre-startedafterapause.
Masterserversnowalsotrackupthepointuptowhichtheyhavesentchangestoclientsforreplication.Thisinformationcanbeusedtodeterminethepointofdatathatreplicationclientshavereceivedfromthemaster,andifandhowfarapproximatelytheylagbehind.
Finally,restartingthereplicationapplieronaslaveserverhasbeenmademorerobustincasetheapplierwasstoppedwhiletherewerependingtransactionsonthemasterserver,andre-startingthereplicationapplierneedstorestorethestateofthesetransactions.
Clienttools
Thefilenamesindumpscreatedbyarangodumpnowcontainnotonlythenameofthedumpedcollection,butalsoanadditional32-digithashvalue.Thisisdonetopreventoverwritingdumpfilesincase-insensitivefilesystemswhenthereexistmultiplecollectionswiththesamename(butwithdifferentcases).
Forexample,ifadatabasehadtwocollectionstestandTest,previousversionsofarangodumpcreatedthefollowingfiles:
test.structure.jsonandtest.data.jsonforcollectiontestTest.structure.jsonandTest.data.jsonforcollectionTest
Thisdidnotworkincase-insensitivefilesystems,becausethefilesforthesecondcollectionwouldhaveoverwrittenthefilesofthefirst.arangodumpin2.7willcreatetheuniquefilesinthiscase,byappendingthe32-digithashvaluetothecollectionnameinallcase.Thesefilenameswillbeunambiguousevenincase-insensitivefilesystems.
Miscellaneouschanges
WhatsNewin2.7
546
Bettercontrol-Csupportinarangosh
WhenCTRL-Cispressedinarangosh,itwillnowabortthelocallyrunningcommand(ifany).Ifnocommandwasrunning,pressingCTRL-Cwillprinta Cfirst.PressingCTRL-Cagainwillthenquitarangosh.
CTRL-Ccanalsobeusedtoresetthecurrentpromptwhileenteringcomplexnestedobjectswhichspanmultipleinputlines.
CTRL-CsupporthasbeenaddedtotheArangoShellversionsbuiltwithReadline-support(LinuxandMacOSonly).TheWindowsversionofArangoDBusesadifferentlibraryforhandlinginput,andsupportforCTRL-Chasnotbeenaddedthereyet.
Start/stop
LinuxstartupscriptsandsystemdconfigurationforarangodnowtrytoadjusttheNOFILE(numberofopenfiles)limitsfortheprocess.Thelimitvalueissetto131072(128k)whenArangoDBisstartedviastart/stopcommands.
ThiswillpreventarangodrunningoutofavailablefiledescriptorsincaseofmanyparallelHTTPconnectionsorlargecollectionswithmanydatafiles.
Additionally,whenArangoDBisstarted/stoppedmanuallyviathestart/stopcommands,themainprocesswillwaitforupto10secondsafteritforksthesupervisorandarangodchildprocesses.Ifthestartupfailswithinthatperiod,thestart/stopscriptwillfailwithanon-zeroexitcode,allowinganyinvokingscriptstohandlethiserror.Previousversionsalwaysreturnedanexitcodeof0,evenwhenarangodcouldn'tbestarted.
Ifthestartupofthesupervisororarangodisstillongoingafter10seconds,themainprogramwillstillreturnwithexitcode0inordertonotblockanyscripts.Thelimitof10secondsisarbitrarybecausethetimerequiredforanarangodstartupisnotknowninadvance.
Non-sparselogfiles
WALlogfilesanddatafilescreatedbyarangodarenownon-sparse.ThispreventsSIGBUSsignalsbeingraisedwhenamemory-mappedregionbackedbyasparsedatafilewasaccessedandthememoryregionwasnotactuallybackedbydisk,forexamplebecausethediskranoutofspace.
arangodnowalwaysfullyallocatesthediskspacerequiredforalogfileordatafilewhenitcreatesone,sothememoryregioncanalwaysbebackedbydisk,andmemorycanbeaccessedwithoutSIGBUSbeingraised.
WhatsNewin2.7
547
IncompatiblechangesinArangoDB2.7ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB2.7,andadjustanyclientprogramsifnecessary.
AQLchanges
DISTINCTisnowakeywordinAQL.
AQLqueriesthatuseDISTINCT(inlower,upperormixedcase)asanidentifier(i.e.asavariable,acollectionnameorafunctionname)willstopworking.Tomakesuchqueriesworkingagain,eachoccurrenceofDISTINCTinanAQLqueryshouldbeenclosedinbackticks.ThiswillturnDISTINCTfromakeywordintoanidentifieragain.
TheAQLfunctionSKIPLIST()hasbeenremovedinArangoDB2.7.ThisfunctionwasdeprecatedinArangoDB2.6.Itwasaleft-overfromtimeswhenthequeryoptimizerwasn'tabletouseskiplistindexestogetherwithfilters,skipandlimitvalues.Sincethisissuebeenfixedsinceversion2.3,thereisnoAQLreplacementfunctionforSKIPLIST.QueriesthatusetheSKIPLISTfunctioncanbefixedbyusingtheusualcombinationofFOR,FILTERandLIMIT,e.g.
FORdocIN@@collection
FILTERdoc.value>=@value
SORTdoc.valueDESC
LIMIT1
RETURNdoc
Foxxchanges
Bundlingandcompilation
TheassetspropertyisnolongersupportedinFoxxmanifestsandisscheduledtoberemovedinafutureversionofArangoDB.Thefilespropertycanstillbeusedtoservestaticassetsbutitisrecommendedtouseseparatetoolingtocompileandbundleyourassets.
Manifestscripts
Thepropertiessetupandteardownhavebeenmovedintothescriptspropertymap:
Before:
{
...
"setup":"scripts/setup.js",
"teardown":"scripts/teardown.js"
}
After:
{
...
"scripts":{
"setup":"scripts/setup.js",
"teardown":"scripts/teardown.js"
}
}
FoxxQueues
Function-basedFoxxQueuejobtypesarenolongersupported.Tolearnabouthowyoucanusethenewscript-basedjobtypesfollowtheupdatedrecipeinthecookbook.
Incompatiblechangesin2.7
548
FoxxSessions
ThejwtandtypeoptionshavebeenremovedfromtheactivateSessionsAPI.
IfyouwanttoreplicatethebehaviorofthejwtoptionyoucanusetheJWTfunctionsinthecryptomodule.AJWT-basedsessionstoragethatdoesn'twritesessionstothedatabaseisavailableasthesessions-jwtappintheFoxxappstore.
Thesessiontypeisnowinferredfromthepresenceofthecookieorheaderoptions(allowingyoutoenablesupportforboth).Ifyouwanttousethedefaultsettingsforcookieorheaderyoucanpassthevaluetrueinstead.
ThesessionStorageAppoptionhasbeenremovedinfavourofthesessionStorageoption.
Before:
varFoxx=require('org/arangodb/foxx');
varctrl=newFoxx.Controller(applicationContext);
ctrl.activateSessions({
sessionStorageApp:'some-sessions-app',
type:'cookie'
});
After:
ctrl.activateSessions({
sessionStorage:applicationContext.dependencies.sessions.sessionStorage,
cookie:true
});
Requestmodule
Themoduleorg/arangodb/requestusesaninternallibraryfunctionforsendingHTTPrequests.ThislibraryfunctionallyunconditionallysetanHTTPheaderAccept-Encoding:gzipinalloutgoingHTTPrequests,withoutclientcodehavingtosetthisheaderexplicitly.
Thishasbeenfixedin2.7,soAccept-Encoding:gzipisnotsetautomaticallyanymore.AdditionallytheheaderUser-Agent:ArangoDBisnotsetautomaticallyeither.Ifclientapplicationsrelyontheseheadersbeingsent,theyarefreetoadditwhenconstructingrequestsusingtherequestmodule.
Theinternal.download()functionisalsoaffectedbythischange.Again,theheadercanbeaddedhereifrequiredbypassingitviaaheaderssub-attributeinthethirdparameter(options)tothisfunction.
arangodump/backupsThefilenamesindumpscreatedbyarangodumpnowcontainnotonlythenameofthedumpedcollection,butalsoanadditional32-digithashvalue.Thisisdonetopreventoverwritingdumpfilesincase-insensitivefilesystemswhenthereexistmultiplecollectionswiththesamename(butwithdifferentcases).
Thischangeleadstochangedfilenamesindumpscreatedbyarangodump.Ifanyclientscriptsdependonthefilenamesinthedumpoutputdirectorybeingequaltothecollectionnameplusoneofthesuffixes.structure.jsonand.data.json,theyneedtobeadjusted.
StartingwithArangoDB2.7,thefilenameswillcontainanunderscoreplusthe32-digitMD5value(representedinhexadecimalnotation)ofthecollectionname.
Forexample,whenarangodumpdumpsdataoftwocollectionstestandTest,thefilenamesinpreviousversionsofArangoDBwere:
test.structure.json(definitionsforcollectiontest)test.data.json(dataforcollectiontest)Test.structure.json(definitionsforcollectionTest)Test.data.json(dataforcollectionTest)
In2.7,thefilenameswillbe:
test_098f6bcd4621d373cade4e832627b4f6.structure.json(definitionsforcollectiontest)
Incompatiblechangesin2.7
549
test_098f6bcd4621d373cade4e832627b4f6.data.json(dataforcollectiontest)Test_0cbc6611f5540bd0809a388dc95a615b.structure.json(definitionsforcollectionTest)Test_0cbc6611f5540bd0809a388dc95a615b.data.json(dataforcollectionTest)
Starting/stopping
Whenstartingarangod,theserverwillnowdroptheprocessprivilegestothespecifiedvaluesinoptions--server.uidand--server.gidinstantlyafterparsingthestartupoptions.
Thatmeanswheneither--server.uidor--server.gidareset,theprivilegechangewillhappenearlier.Thismaypreventbindingtheservertoanendpointwithaportnumberlowerthan1024ifthearangodbuserhasnoprivilegesforthat.PreviousversionsofArangoDBchangedtheprivilegeslater,sosomestartupactionswerestillcarriedoutundertheinvokinguser(i.e.likelyrootwhenstartedviainit.dorsystemscripts)andespeciallybindingtolowportnumberswasstillpossiblethere.
Thedefaultprivilegesforuserarangodbwillnotbesufficientforbindingtoportnumberslowerthan1024.TohaveanArangoDB2.7bindtoaportnumberlowerthan1024,itneedstobestartedwitheitheradifferentprivilegeduser,ortheprivilegesofthearangodbuserhavetoraisedmanuallybeforehand.
Additionally,LinuxstartupscriptsandsystemdconfigurationforarangodnowwilladjusttheNOFILE(numberofopenfiles)limitsfortheprocess.Thelimitvalueissetto131072(128k)whenArangoDBisstartedviastart/stopcommands.Thegoalofthischangeistopreventarangodfromrunningoutofavailablefiledescriptorsforsocketconnectionsanddatafiles.
Connectionhandling
arangodwillnowactuallycloselingeringclientconnectionswhenidleforatleastthedurationspecifiedinthe--server.keep-alive-timeoutstartupoption.
InpreviousversionsofArangoDB,idleconnectionswerenotclosedbytheserverwhenthetimeoutwasreachedandtheclientwasstillconnected.Nowtheconnectionisproperlyclosedbytheserverincaseoftimeout.Clientapplicationsrelyingontheoldbehaviormaynowneedtoreconnecttotheserverwhentheiridleconnectionstimeoutandgetclosed(note:connectionsbeingidleforalongtimemaybeclosedbytheOSorfirewallsanyway-clientapplicationsshouldbeawareofthatandtrytoreconnect).
Optionchanges
Configureoptionsremoved
Thefollowingoptionsforconfigurehavebeenremovedbecausetheywereunusedorexotic:
--enable-timings
--enable-figures
Startupoptionsadded
Thefollowingconfigurationoptionshavebeenaddedin2.7:
--database.query-cache-max-results:setsthemaximumnumberofresultsinAQLqueryresultcacheperdatabase--database.query-cache-mode:setsthemodefortheAQLqueryresultscache.Possiblevaluesareon,offanddemand.Thedefaultvalueisoff
Miscellaneouschanges
Simplequeries
Manysimplequeriesprovideaskip()functionthatcanbeusedtoskipoveracertainnumberofdocumentsintheresult.ThisfunctionallowedspecifyingnegativeoffsetsinpreviousversionsofArangoDB.Specifyinganegativeoffsetledtothequeryresultbeingiteratedinreverseorder,soskippingwasperformedfromthebackoftheresult.Asmostsimplequeriesdonotprovideaguaranteedresultorder,
Incompatiblechangesin2.7
550
skippingfromthebackofaresultwithunspecificorderseemsaratherexoticusecaseandwasremovedtoincreaseconsistencywith
AQL,whichalsodoesnotprovidenegativeskipvalues.
NegativeskipvaluesweredeprecatedinArangoDB2.6.
TasksAPI
TheundocumentedfunctionaddJob()hasbeenremovedfromtheorg/arangodb/tasksmoduleinArangoDB2.7.
RuntimeendpointsmanipulationAPI
ThefollowingHTTPRESTAPImethodsforruntimemanipulationofserverendpointshavebeenremovedinArangoDB2.7:
POST/_api/endpoint:todynamicallyaddanendpointwhiletheserverwasrunningDELETE/_api/endpoint:todynamicallyremoveanendpointwhiletheserverwasrunning
ThischangealsoaffectstheequivalentJavaScriptendpointmanipulationmethodsavailableinFoxx.ThefollowingfunctionshavebeenremovedinArangoDB2.7:
db._configureEndpoint()
db._removeEndpoint()
Incompatiblechangesin2.7
551
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.6.ArangoDB2.6alsocontainsseveralbugfixesthatarenotlistedhere.Foralistofbugfixes,pleaseconsulttheCHANGELOG.
APIsadded
Batchdocumentremovalandlookupcommands
Thefollowingcommandshavebeenaddedforcollectionobjects:
collection.lookupByKeys(keys)collection.removeByKeys(keys)
Thesecommandscanbeusedtoperformmulti-documentlookupandremovaloperationsefficientlyfromtheArangoShell.Theargumenttotheseoperationsisanarrayofdocumentkeys.
ThesecommandscanalsobeusedviatheHTTPRESTAPI.Theirendpointsare:
PUT/_api/simple/lookup-by-keysPUT/_api/simple/remove-by-keys
CollectionexportHTTPRESTAPI
ArangoDBnowprovidesadedicatedcollectionexportAPI,whichcantakesnapshotsofentirecollectionsmoreefficientlythanthegeneral-purposecursorAPI.TheexportAPIisusefultotransferthecontentsofanentirecollectiontoaclientapplication.Itprovidesoptionalfilteringonspecificattributes.
TheexportAPIisavailableatendpointPOST/_api/export?collection=....TheAPIhasthesamereturnvaluestructureasthealreadyestablishedcursorAPI(POST/_api/cursor).
AnintroductiontotheexportAPIisgiveninthisblogpost:http://jsteemann.github.io/blog/2015/04/04/more-efficient-data-exports/
AQLimprovements
EDGESAQLFunction
TheAQLfunctionEDGESgotanewfifthoptionalparameter,whichmustbeanobjectifspecified.Rightnowonlyoneoptionisavailableforit:
includeVerticesthisisabooleanparameterthatallowstomodifytheresultofEDGES().ThedefaultvalueforincludeVerticesisfalse,whichdoesnothaveanyeffect.Settingittotruewillmodifytheresult,suchthatalsotheconnectedverticesarereturnedalongwiththeedges:
{vertex:<vertexDocument>,edge:<edgeDocument>}
SubqueryoptimizationsforAQLqueries
Thisoptimizationavoidscopyingintermediateresultsintosubqueriesthatarenotrequiredbythesubquery.
Abriefdescriptioncanbefoundhere:http://jsteemann.github.io/blog/2015/05/04/subquery-optimizations/
ReturnvalueoptimizationforAQLqueries
Thisoptimizationavoidscopyingthefinalqueryresultinsidethequery'smainReturnNode.
Abriefdescriptioncanbefoundhere:http://jsteemann.github.io/blog/2015/05/04/return-value-optimization-for-aql/
WhatsNewin2.6
552
SpeedupAQLqueriescontainingbigINlistsforindexlookups
INlistsusedforindexlookupshadperformanceissuesinpreviousversionsofArangoDB.Theseissueshavebeenaddressedin2.6sousingbiggerINlistsforfilteringismuchfaster.
Abriefdescriptioncanbefoundhere:http://jsteemann.github.io/blog/2015/05/07/in-list-improvements/
AddedalternativeimplementationforAQLCOLLECT
Thealternativemethodusesahashtableforgroupinganddoesnotrequireitsinputelementstobesorted.ItwillbetakenintoaccountbytheoptimizerforCOLLECTstatementsthatdonotuseanINTOclause.
IncaseaCOLLECTstatementcanusethehashtablevariant,theoptimizerwillcreateanextraplanforitatthebeginningoftheplanningphase.Inthisplan,noextraSORTnodewillbeaddedinfrontoftheCOLLECTbecausethehashtablevariantofCOLLECTdoesnotrequiresortedinput.Instead,aSORTnodewillbeaddedafterittosortitsoutput.ThisSORTnodemaybeoptimizedawayagaininlaterstages.Ifthesortorderoftheresultisirrelevanttotheuser,addinganextraSORTnullafterahashCOLLECToperationwillallowtheoptimizertoremovethesortsaltogether.
InadditiontothehashtablevariantofCOLLECT,theoptimizerwillmodifytheoriginalplantousetheregularCOLLECTimplementation.Asthisimplementationrequiressortedinput,theoptimizerwillinsertaSORTnodeinfrontoftheCOLLECT.ThisSORTnodemaybeoptimizedawayinlaterstages.
Thecreatedplanswillthenbeshippedthroughtheregularoptimizationpipeline.Intheend,theoptimizerwillpicktheplanwiththelowestestimatedtotalcostasusual.Thehashtablevariantdoesnotrequireanup-frontsortoftheinput,andwillthusbepreferredovertheregularCOLLECTiftheoptimizerestimatesmanyinputelementsfortheCOLLECTnodeandcannotuseanindextosortthem.
TheoptimizercanbeexplicitlytoldtousetheregularsortedvariantofCOLLECTbysuffixingaCOLLECTstatementwithOPTIONS{"method":"sorted"}.ThiswilloverridetheoptimizerguessworkandonlyproducethesortedvariantofCOLLECT.
AblogpostonthenewCOLLECTimplementationcanbefoundhere:http://jsteemann.github.io/blog/2015/04/22/collecting-with-a-hash-table/
Simplifiedreturnvaluesyntaxfordata-modificationAQLqueries
ArangoDB2.4sinceversionallowstoreturnresultsfromdata-modificationAQLqueries.Thesyntaxforthiswasquitelimitedandverbose:
FORiIN1..10
INSERT{value:i}INtest
LETinserted=NEW
RETURNinserted
TheLETinserted=NEWRETURNinsertedwasrequiredliterallytoreturntheinserteddocuments.Nocalculationscouldbemadeusingtheinserteddocuments.
Thisisnowmoreflexible.Afteradata-modificationclause(e.g.INSERT,UPDATE,REPLACE,REMOVE,UPSERT)therecanfollowanynumberofLETcalculations.Thesecalculationscanrefertothepseudo-valuesOLDandNEWthatarecreatedbythedata-modificationstatements.
Thisallowsreturningprojectionsofinsertedorupdateddocuments,e.g.:
FORiIN1..10
INSERT{value:i}INtest
RETURN{_key:NEW._key,value:i}
Stillnoteveryconstructisallowedafteradata-modificationclause.Forexample,nofunctionscanbecalledthatmayaccessdocuments.
Moreinformationcanbefoundhere:http://jsteemann.github.io/blog/2015/03/27/improvements-for-data-modification-queries/
AddedAQLUPSERTstatement
WhatsNewin2.6
553
ThisaddsanUPSERTstatementtoAQLthatisacombinationofbothINSERTandUPDATE/REPLACE.TheUPSERTwillsearchforamatchingdocumentusingauser-providedexample.Ifnodocumentmatchestheexample,theinsertpartoftheUPSERTstatementwillbeexecuted.Ifthereisamatch,theupdate/replacepartwillbecarriedout:
UPSERT{page:'index.html'}/*searchexample*/
INSERT{page:'index.html',pageViews:1}/*insertpart*/
UPDATE{pageViews:OLD.pageViews+1}/*updatepart*/
INpageViews
UPSERTcanbeusedwithanUPDATEorREPLACEclause.TheUPDATEclausewillperformapartialupdateofthefounddocument,whereastheREPLACEclausewillreplacethefounddocumententirely.TheUPDATEorREPLACEpartscanrefertothepseudo-valueOLD,whichcontainsallattributesofthefounddocument.
UPSERTstatementscanoptionallyreturnvalues.Inthefollowingquery,thereturnattributefoundwillreturnthefounddocumentbeforetheUPDATEwasapplied.Ifnodocumentwasfound,foundwillcontainavalueofnull.Theupdatedresultattributewillcontaintheinserted/updateddocument:
UPSERT{page:'index.html'}/*searchexample*/
INSERT{page:'index.html',pageViews:1}/*insertpart*/
UPDATE{pageViews:OLD.pageViews+1}/*updatepart*/
INpageViews
RETURN{found:OLD,updated:NEW}
AmoredetaileddescriptionofUPSERTcanbefoundhere:http://jsteemann.github.io/blog/2015/03/27/preview-of-the-upsert-command/
Miscellaneouschanges
WhenerrorsoccurinsideAQLuserfunctions,theerrormessagewillnowcontainastacktrace,indicatingthelineofcodeinwhichtheerroroccurred.ThisshouldmakedebuggingAQLuserfunctionseasier.
WebAdminInterfaceArangoDB'sbuilt-inwebinterfacenowusessessions.Sessioninformationisstoredincookies,soclientsusingthewebinterfacemustacceptcookiesinordertouseit.
Thenewstartupoption--server.session-timeoutcanbeusedforadjustingthesessionlifetime.
TheAQLeditorinthewebinterfacenowprovidesanexplainfunctionality,whichcanbeusedforinspectingandperformance-tuningAQLqueries.ThequeryexecutiontimeisnowalsodisplayedintheAQLeditor.
Foxxappsthatrequireconfigurationoraremissingdependenciesarenowindicatedintheappoverviewanddetails.
Foxximprovements
ConfigurationandDependencies
Foxxappmanifestscannowdefineconfigurationoptions,aswellasdependenciesonotherFoxxapps.
AnintroductiontoFoxxconfigurationscanbefoundintheblog:https://www.arangodb.com/2015/05/reusable-foxx-apps-with-configurations/
AndtheblogpostonFoxxdependenciescanbefoundhere:https://www.arangodb.com/2015/05/foxx-dependencies-for-more-composable-foxx-apps/
MochaTests
YoucannowwritetestsforyourFoxxappsusingtheMochatestingframework:https://www.arangodb.com/2015/04/testing-foxx-mocha/
ArecipeforwritingtestsforyourFoxxappscanbefoundinthecookbook:https://docs.arangodb.com/2.8/cookbook/FoxxTesting.html
WhatsNewin2.6
554
APIDocumentation
TheAPIdocumentationhasbeenupdatedtoSwagger2.YoucannowalsomountAPIdocumentationinyourownFoxxapps.
Alsoseetheblogpostintroducingthisfeature:https://www.arangodb.com/2015/05/document-your-foxx-apps-with-swagger-2/
CustomScriptsandFoxxQueue
InadditiontotheexistingsetupandteardownscriptsyoucannowdefinecustomscriptsinyourFoxxmanifestandinvoketheseusingthewebadmininterfaceortheFoxxmanagerCLI.Thesescriptscannowalsotakepositionalargumentsandexportreturnvalues.
JobtypesfortheFoxxQueuecannowbedefinedasascriptnameandappmountpathallowingtheuseofFoxxscriptsasjobtypes.Thepre-2.6jobtypesareknowntocauseissueswhenrestartingtheserverandareerror-prone;westronglyrecommendedconvertinganyexistingjobtypestothenewformat.
Clienttools
Thedefaultconfigurationvaluefortheoption--server.request-timeoutwasincreasedfrom300to1200secondsforallclienttools(arangosh,arangoimp,arangodump,arangorestore).
Thedefaultconfigurationvaluefortheoption--server.connect-timeoutwasincreasedfrom3to5secondsforclienttools(arangosh,arangoimp,arangodump,arangorestore).
Arangorestore
Theoption--create-databasewasaddedforarangorestore.
Settingthisoptiontotruewillnowcreatethetargetdatabaseifitdoesnotexist.Whencreatingthetargetdatabase,theusernameandpasswordspassedtoarangorestorewillbeusedtocreateaninitialuserforthenewdatabase.
Thedefaultvalueforthisoptionisfalse.
Arangoimp
Arangoimpcannowoptionallyupdateorreplaceexistingdocuments,providedtheimportdatacontainsdocumentswith_keyattributes.
Previously,theimportcouldbeusedforinsertingnewdocumentsonly,andre-insertingadocumentwithanexistingkeywouldhavefailedwithauniquekeyconstraintviolatederror.
Thebehaviorofarangoimp(insert,update,replaceonduplicatekey)cannowbecontrolledwiththeoption--on-duplicate.Theoptioncanhaveoneofthefollowingvalues:
error:whenauniquekeyconstrainterroroccurs,donotimportorupdatethedocumentbutreportanerror.Thisisthedefault.
update:whenauniquekeyconstrainterroroccurs,tryto(partially)updatetheexistingdocumentwiththedataspecifiedintheimport.Thismaystillfailifthedocumentwouldviolatesecondaryuniqueindexes.Onlytheattributespresentintheimportdatawillbeupdatedandotherattributesalreadypresentwillbepreserved.ThenumberofupdateddocumentswillbereportedintheupdatedattributeoftheHTTPAPIresult.
replace:whenauniquekeyconstrainterroroccurs,trytofullyreplacetheexistingdocumentwiththedataspecifiedintheimport.Thismaystillfailifthedocumentwouldviolatesecondaryuniqueindexes.ThenumberofreplaceddocumentswillbereportedintheupdatedattributeoftheHTTPAPIresult.
ignore:whenauniquekeyconstrainterroroccurs,ignorethiserror.Therewillbenoinsert,updateorreplacefortheparticulardocument.IgnoreddocumentswillbereportedseparatelyintheignoredattributeoftheHTTPAPIresult.
Thedefaultvalueiserror.
Afewexamplesforusingarangoimpwiththe--on-duplicateoptioncanbefoundhere:http://jsteemann.github.io/blog/2015/04/14/updating-documents-with-arangoimp/
WhatsNewin2.6
555
Miscellaneouschanges
SomeLinux-basedArangoDBpackagesarenowusingtcmallocformemoryallocator.
UpgradedICUlibrarytoversion54.Thisincreasesperformanceinmanyplaces.
Allowtosplitanedgeindexintobucketswhichareresizedindividually.Thedefaultvalueis1,resemblingthepre-2.6behavior.Usingmultiplebucketswillleadtotheindexentriesbeingdistributedtotheindividualbuckets,witheachbucketbeingresponsibleonlyforafractionofthetotalindexentries.Usingmultiplebucketsmayleadtomorefrequentbutmuchfasterindexbucketresizes,andisrecommendedforbiggeredgecollections.
Defaultconfigurationvalueforoption--server.backlog-sizewaschangedfrom10to64.
Defaultconfigurationvalueforoption--database.ignore-datafile-errorswaschangedfromtruetofalse
Fulltextindexcannowindextextvaluescontainedindirectsub-objectsoftheindexedattribute.
PreviousversionsofArangoDBonlyindexedtheattributevalueifitwasastring.Sub-attributesoftheindexattributewereignoredwhenfulltextindexing.
Now,iftheindexattributevalueisanobject,theobject'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.Iftheindexattributevalueisanarray,thearray'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.
Forexample,withafulltextindexpresentonthetranslationsattribute,thefollowingtextvalueswillnowbeindexed:
varc=db._create("example");
c.ensureFulltextIndex("translations");
c.insert({translations:{en:"fox",de:"Fuchs",fr:"renard",ru:"лиса"}});
c.insert({translations:"FoxistheEnglishtranslationoftheGermanwordFuchs"});
c.insert({translations:["ArangoDB","document","database","Foxx"]});
c.fulltext("translations","лиса").toArray();//returnsonlyfirstdocument
c.fulltext("translations","Fox").toArray();//returnsfirstandseconddocuments
c.fulltext("translations","prefix:Fox").toArray();//returnsallthreedocuments
Addedconfigurationoption--server.foxx-queues-poll-interval
ThisstartupoptioncontrolsthefrequencywithwhichtheFoxxqueuesmanagerischeckingthequeue(orqueues)forjobstobeexecuted.
Thedefaultvalueis1second.Loweringthisvaluewillresultinthequeuemanagerwakingupandcheckingthequeuesmorefrequently,whichmayincreaseCPUusageoftheserver.WhennotusingFoxxqueues,thisvaluecanberaisedtosavesomeCPUtime.
Addedconfigurationoption--server.foxx-queues
ThisstartupoptioncontrolswhethertheFoxxqueuemanagerwillcheckqueueandjobentriesinthe_systemdatabaseonly.RestrictingtheFoxxqueuemanagertothe_systemdatabasewillleadtothequeuemanagerhavingtocheckonlythequeuescollectionofasingledatabase,whereasmakingitcheckthequeuesofalldatabasesmightresultinmoreworktobedoneandmoreCPUtimetobeusedbythequeuemanager.
WhatsNewin2.6
556
IncompatiblechangesinArangoDB2.6ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB2.6,andadjustanyclientprogramsifnecessary.
Requirements
ArangoDB'sbuilt-inwebinterfacenowusescookiesforsessionmanagement.Sessioninformationidsarestoredincookies,soclientsusingthewebinterfacemustacceptcookiesinordertologinanduseit.
Foxxchanges
FoxxQueues
FoxxQueuejobtypedefinitionswerepreviouslybasedonfunctionsandhadtoberegisteredbeforeuse.Duetochangesin2.5thisresultedinproblemswhenrestartingtheserverordefiningjobtypesincorrectly.
Function-basedjobtypeshavebeendeprecatedin2.6andwillberemovedentirelyin2.7.
Inordertoconvertexistingfunction-basedjobtypestothenewscript-basedjobtypes,createcustomscriptsinyourFoxxappandreferencethembytheirnameandthemountpointoftheapptheyaredefinedin.OfficialjobtypesfromtheFoxxappstorecanbeupgradedbyupgradingfromthe1.xversiontothe2.xversionofthesameapp.
Inordertoupgradequeuedjobstothenewjobtypes,youneedtoupdatethetypepropertyoftheaffectedjobsinthedatabase's_jobssystemcollection.Inordertoseethecollectioninthewebinterfaceyouneedtoenablethecollectiontype"System"inthecollectionlistoptions.
Example:
Before:"type":"mailer.postmark"
After:"type":{"name":"mailer","mount":"/my-postmark-mailer"}
FoxxSessions
Theoptionsjwtandtypeofthecontrollermethodcontroller.activateSessionshavebeendeprecatedin2.6andwillberemovedentirelyin2.7.
IfyouwanttousepureJWTsessions,youcanusethesessions-jwtFoxxappfromtheFoxxappstore.
IfyouwanttouseyourownJWT-basedsessions,youcanusetheJWTfunctionsinthecryptomoduledirectly.
Insteadofusingthetypeoptionyoucanjustusethecookieandheaderoptionsontheirown,whichbothnowacceptthevaluetruetoenablethemwiththeirdefaultconfigurations.
TheoptionsessionStorageApphasbeenrenamedtosessionStorageandnowalsoacceptssessionstoragesdirectly.TheoldoptionsessionStorageAppwillberemovedentirelyin2.7.
Libraries
ThebundledversionofthejoilibraryusedinFoxxwasupgradedtoversion6.0.8.ThismayaffectFoxxapplicationsthatdependonthelibrary.
AQLchanges
AQLLENGTHfunction
Incompatiblechangesin2.6
557
ThereturnvalueoftheAQLLENGTHfunctionwaschangedifLENGTHisappliedonnullorabooleanvalue:
LENGTH(null)nowreturns0.InpreviousversionsofArangoDB,thisreturned4.
LENGTH(false)nowreturns0.InpreviousversionsofArangoDB,thereturnvaluewas5.
LENGTH(true)nowreturns1.InpreviousversionsofArangoDB,thereturnvaluewas4.
AQLgraphfunctions
In2.6thegraphfunctionsdidundergoaperformancelifting.Duringthisprocesswehadtoadopttheresultformatandtheoptionsforsomeofthem.ManygraphfunctionsnowhaveanoptionincludeDatawhichallowstotriggeriftheresultofthisfunctionshouldcontainfullyextracteddocumentsincludeData:trueoronlythe_idvaluesincludeData:false.Inmostusecasesthe_idissufficienttocontinueandtheextractionofdataisanunnecessaryoperation.TheAQLfunctionssupportingthisadditionaloptionare:
SHORTEST_PATHNEIGHBORSGRAPH_SHORTEST_PATHGRAPH_NEIGHBORSGRAPH_EDGES
FurthermoretheresultSHORTEST_PATHhaschanged.Theoldformatreturnedalistofallverticesonthepath.Optionallyitcouldincludeeachsub-pathforthesevertices.Allofthedocumentswerefullyextracted.Example:
[
{
vertex:{
_id:"vertex/1",
_key:"1",
_rev:"1234"
name:"Alice"
},
path:{
vertices:[
{
_id:"vertex/1",
_key:"1",
_rev:"1234"
name:"Alice"
}
],
edges:[]
}
},
{
vertex:{
_id:"vertex/2",
_key:"2",
_rev:"5678"
name:"Bob"
},
path:{
vertices:[
{
_id:"vertex/1",
_key:"1",
_rev:"1234"
name:"Alice"
},{
_id:"vertex/2",
_key:"2",
_rev:"5678"
name:"Bob"
}
],
edges:[
{
_id:"edge/1",
_key:"1",
_rev:"9876",
type:"loves"
Incompatiblechangesin2.6
558
}
]
}
}
]
Thenewversionismorecompact.EachSHORTEST_PATHwillonlyreturnonedocumenthavingtheattributesvertices,edges,distance.Thedistanceiscomputedtakingintoaccountthegivenweight.OptionallythedocumentscanbeextractedwithincludeData:trueExample:
{
vertices:[
"vertex/1",
"vertex/2"
],
edges:[
"edge/1"
],
distance:1
}
ThenextfunctionthatreturnsadifferentformatisNEIGHBORS.Since2.5itreturnedanobjectwithedgeandvertexforeachconnectededge.Example:
[
{
vertex:{
_id:"vertex/2",
_key:"2",
_rev:"5678"
name:"Bob"
},
edge:{
_id:"edge/1",
_key:"1",
_rev:"9876",
type:"loves"
}
}
]
With2.6itwillonlyreturnthevertexdirectly,againusingincludeData:true.Bydefaultitwillreturnadistinctsetofneighbors,usingtheoptiondistinct:falsewillincludethesamevertexforeachedgepointingtoit.
Example:
[
"vertex/2"
]
FunctionandAPIchanges
Graphmeasurementsfunctions
AllgraphmeasurementsfunctionsinJavaScriptmodulegeneral-graphthatcalculatedasinglefigurepreviouslyreturnedanarraycontainingjustthefigure.Nowthesefunctionswillreturnthefiguredirectlyandnotputitinsideanarray.
Theaffectedfunctionsare:
graph._absoluteEccentricity
graph._eccentricity
graph._absoluteCloseness
graph._closeness
graph._absoluteBetweenness
Incompatiblechangesin2.6
559
graph._betweenness
graph._radius
graph._diameter
Clientprogramscallingthesefunctionsshouldbeadjustedsotheyprocessthescalarvaluereturnedbythefunctioninsteadofthepreviousarrayvalue.
CursorAPI
AbatchSizevalue0isnowdisallowedwhencallingthecursorAPIviaHTTPPOST/_api/cursor.
TheHTTPRESTAPIPOST/_api/cursordoesnotacceptabatchSizeparametervalueof0anylonger.Abatchsizeof0nevermademuchsense,butpreviousversionsofArangoDBdidnotcheckforthisvalue.NowcreatingacursorusingabatchSizevalue0willresultinanHTTP400errorresponse.
DocumentURLsreturned
TheRESTAPImethodGET/_api/document?collection=...(thatmethodwillreturnpartialURLstoalldocumentsinthecollection)willnowproperlyprefixdocumentaddressURLswiththecurrentdatabasename.
PreviousversionsofArangoDBreturnedtheURLsstartingwith/_api/butwithoutthecurrentdatabasename,e.g./_api/document/mycollection/mykey.Startingwith2.6,theresponseURLswillincludethedatabasenameaswell,e.g./_db/_system/_api/document/mycollection/mykey.
Fulltextindexing
Fulltextindexeswillnowalsoindextextvaluescontainedindirectsub-objectsoftheindexedattribute.
PreviousversionsofArangoDBonlyindexedtheattributevalueifitwasastring.Sub-attributesoftheindexattributewereignoredwhenfulltextindexing.
Now,iftheindexattributevalueisanobject,theobject'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.Iftheindexattributevalueisanarray,thearray'svalueswilleachbeincludedinthefulltextindexiftheyarestrings.
Deprecatedserverfunctionality
Simplequeries
Thefollowingsimplequeryfunctionsarenowdeprecated:
collection.nearcollection.withincollection.geocollection.fulltextcollection.rangecollection.closedRange
ThisalsoleadtothefollowingRESTAPImethodsbeingdeprecatedfromnowon:
PUT/_api/simple/nearPUT/_api/simple/withinPUT/_api/simple/fulltextPUT/_api/simple/range
ItisrecommendedtoreplacecallstothesefunctionsorAPIswithequivalentAQLqueries,whicharemoreflexiblebecausetheycanbecombinedwithotheroperations:
FORdocINNEAR(@@collection,@latitude,@longitude,@limit)
RETURNdoc
FORdocINWITHIN(@@collection,@latitude,@longitude,@radius,@distanceAttributeName)
RETURNdoc
Incompatiblechangesin2.6
560
FORdocINFULLTEXT(@@collection,@attributeName,@queryString,@limit)
RETURNdoc
FORdocIN@@collection
FILTERdoc.value>=@left&&doc.value<@right
LIMIT@skip,@limit
RETURNdoc`
TheabovesimplequeryfunctionsandRESTAPImethodsmayberemovedinfutureversionsofArangoDB.
UsingnegativevaluesforSimpleQuery.skip()isalsodeprecated.ThisfunctionalitywillberemovedinfutureversionsofArangoDB.
AQLfunctions
TheAQLSKIPLISTfunctionhasbeendeprecatedbecauseitisobsolete.
ThefunctionwasintroducedinolderversionsofArangoDBwithalesspowerfulqueryoptimizertoretrievedatafromaskiplistindexusingaLIMITclause.
Since2.3thesamegoalcanbeachievedbyusingregularAQLconstructs,e.g.
FORdocIN@@collection
FILTERdoc.value>=@value
SORTdoc.value
LIMIT1
RETURNdoc
Startupoptionchanges
Optionsadded
Thefollowingconfigurationoptionshavebeenaddedin2.6:
--server.session-timeout:allowscontrollingthetimeoutofusersessionsinthewebinterface.Thevalueisspecifiedinseconds.
--server.foxx-queues:controlswhethertheFoxxqueuemanagerwillcheckqueueandjobentries.DisablingthisoptioncanreduceserverloadbutwillpreventjobsaddedtoFoxxqueuesfrombeingprocessedatall.
Thedefaultvalueistrue,enablingtheFoxxqueuesfeature.
--server.foxx-queues-poll-interval:allowsadjustingthefrequencywithwhichtheFoxxqueuesmanagerischeckingthequeue(orqueues)forjobstobeexecuted.
Thedefaultvalueis1second.Loweringthisvaluewillresultinthequeuemanagerwakingupandcheckingthequeuesmorefrequently,whichmayincreaseCPUusageoftheserver.
Note:thisoptiononlyhasaneffectwhen--server.foxx-queuesisnotsettofalse.
Optionsremoved
Thefollowingconfigurationoptionshavebeenremovedin2.6.:
--log.severity:thedocsfor--log.severitymentionedlotsofseverities(e.g.exception,technical,functional,development)butonlyafewseverities(e.g.all,human)wereactuallyused,withhumanbeingthedefaultandallenablingtheadditionalloggingofincomingrequests.
Theoptionpretendedtocontrolalotofthingswhichitactuallydidn't.Additionally,theoption--log.requests-filewasaroundforalongtimealready,alsocontrollingrequestlogging.
Becausethe--log.severityoptioneffectivelydidnotcontrolthatmuch,itwasremoved.Asideeffectofremovingtheoptionisthat2.5installationsstartedwithoption--log.severityallwillnotlogrequestsaftertheupgradeto2.6.Thiscanbeadjustedbysettingthe--log.requests-fileoptioninstead.
Incompatiblechangesin2.6
561
Defaultvalueschanged
Thedefaultvaluesforthefollowingoptionshavechangedin2.6:
--database.ignore-datafile-errors:thedefaultvalueforthisoptionwaschangedfromtruetofalse.
Ifthenewdefaultvalueoffalseisused,thenarangodwillrefuseloadingcollectionsthatcontaindatafileswithCRCmismatchesorothererrors.Acollectionwithdatafileerrorswillthenbecomeunavailable.Thispreventsfollowuperrorsfromhappening.
Theonlywaytoaccesssuchcollectionistousethedatafiledebugger(arango-dfdb)andtrytorepairortruncatethedatafilewithit.
--server.request-timeout:thedefaultvaluewasincreasedfrom300to1200secondsforallclienttools(arangosh,arangoimp,arangodump,arangorestore).
--server.connect-timeout:thedefaultvaluewasincreasedfrom3to5secondsforallclienttools(arangosh,arangoimp,arangodump,arangorestore).
Incompatiblechangesin2.6
562
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.5.ArangoDB2.5alsocontainsseveralbugfixesthatarenotlistedhere.Foralistofbugfixes,pleaseconsulttheCHANGELOG.
V8versionupgrade
Thebuilt-inversionofV8hasbeenupgradedfrom3.29.54to3.31.74.1.ThisallowsactivatingadditionalES6(alsodubbedHarmonyorES.next)featuresinArangoDB,bothintheArangoShellandtheArangoDBserver.Theycanbeusedforscriptingandinserver-sideactionssuchasFoxxroutes,traversalsetc.
ThefollowingadditionalES6featuresbecomeavailableinArangoDB2.5bydefault:
iteratorsandgeneratorstemplatestringsenhancedobjectliteralsenhancednumericliteralsblockscopingwithletandconstantvariablesusingconst(note:constantvariablesrequireusingstrictmode,too)additionalstringmethods(suchasstartsWith,repeatetc.)
Indeximprovements
Sparsehashandskiplistindexes
Hashandskiplistindexescanoptionallybemadesparse.Sparseindexesexcludedocumentsinwhichatleastoneoftheindexattributesiseithernotsetorhasavalueofnull.
Assuchdocumentsareexcludedfromsparseindexes,theymaycontainfewerdocumentsthantheirnon-sparsecounterparts.Thisenablesfasterindexingandcanleadtoreducedmemoryusageincasetheindexedattributedoesoccuronlyinsome,butnotalldocumentsofthecollection.Sparseindexeswillalsoreducethenumberofcollisionsinnon-uniquehashindexesincasenon-existingoroptionalattributesareindexed.
Inordertocreateasparseindex,anobjectwiththeattributesparsecanbeaddedtotheindexcreationcommands:
db.collection.ensureHashIndex(attributeName,{sparse:true});
db.collection.ensureHashIndex(attributeName1,attributeName2,{sparse:true});
db.collection.ensureUniqueConstraint(attributeName,{sparse:true});
db.collection.ensureUniqueConstraint(attributeName1,attributeName2,{sparse:true});
db.collection.ensureSkiplist(attributeName,{sparse:true});
db.collection.ensureSkiplist(attributeName1,attributeName2,{sparse:true});
db.collection.ensureUniqueSkiplist(attributeName,{sparse:true});
db.collection.ensureUniqueSkiplist(attributeName1,attributeName2,{sparse:true});
Notethatinplaceoftheabovespecializedindexcreationcommands,itisrecommendedtousethemoregeneralindexcreationcommandensureIndex:
db.collection.ensureIndex({type:"hash",sparse:true,unique:true,fields:[attributeName]});
db.collection.ensureIndex({type:"skiplist",sparse:false,unique:false,fields:["a","b"]});
Whennotexplicitlyset,thesparseattributedefaultstofalsefornewhashorskiplistindexes.
Thiscausesachangeinbehaviorwhencreatingauniquehashindexwithoutspecifyingthesparseflag:in2.4,uniquehashindexeswereimplicitlysparse,alwaysexcludingnullvalues.Therewasnooptiontocontrolthisbehavior,andsparsitywasneithersupportedfornon-uniquehashindexesnorskiplistsin2.4.Thisimplicitsparsityofuniquehashindexeswasconsideredaninconsistency,andthereforethebehaviorwascleanedupin2.5.Asof2.5,indexeswillonlybecreatedsparseifsparsityisexplicitlyrequested.Existinguniquehashindexesfrom2.4orbeforewillautomaticallybemigratedsotheyarestillsparseaftertheupgradeto2.5.
WhatsNewin2.5
563
Geoindexesareimplicitlysparse,meaningdocumentswithouttheindexedlocationattributeorcontaininginvalidlocationcoordinatevalueswillbeexcludedfromtheindexautomatically.Thisisalsoachangewhencomparedtopre-2.5behavior,whendocumentswithmissingorinvalidcoordinatevaluesmayhavecausederrorsoninsertionwhenthegeoindex'uniqueflagwassetanditsignoreNullflagwasnot.Thiswasconfusingandhasbeenrectifiedin2.5.ThemethodensureGeoConstraint()nowdoesthesameasensureGeoIndex().Furthermore,theattributesconstraint,unique,ignoreNullandsparseflagsarenowcompletelyignoredwhencreatinggeoindexes.
Thesameistrueforfulltextindexes.Thereisnoneedtospecifynon-uniquenessorsparsityforgeoorfulltextindexes.
Assparseindexesmayexcludesomedocuments,theycannotbeusedforeverytypeofquery.Sparsehashindexescannotbeusedtofinddocumentsforwhichatleastoneoftheindexedattributeshasavalueofnull.Forexample,thefollowingAQLquerycannotuseasparseindex,evenifonewascreatedonattributeattr:
FORdocIncollection
FILTERdoc.attr==null
RETURNdoc
Ifthelookupvalueisnon-constant,asparseindexmayormaynotbeused,dependingontheothertypesofconditionsinthequery.Iftheoptimizercansafelydeterminethatthelookupvaluecannotbenull,asparseindexmaybeused.Whenuncertain,theoptimizerwillnotmakeuseofasparseindexinaqueryinordertoproducecorrectresults.
Forexample,thefollowingqueriescannotuseasparseindexonattrbecausetheoptimizerwillnotknowbeforehandwhetherthecomparisonvaluesfordoc.attrwillincludenull:
FORdocIncollection
FILTERdoc.attr==SOME_FUNCTION(...)
RETURNdoc
FORotherINotherCollection
FORdocIncollection
FILTERdoc.attr==other.attr
RETURNdoc
Sparseskiplistindexescanbeusedforsortingiftheoptimizercansafelydetectthattheindexrangedoesnotincludenullforanyoftheindexattributes.
Selectivityestimates
Indexesoftypeprimary,edgeandhashnowprovideselectivityestimates.ThesewillbeusedbytheAQLqueryoptimizerwhendecidingaboutindexusage.Usingselectivityestimatescanleadtofasterqueryexecutionwhenmoreselectiveindexesareused.
TheselectivityestimatesarealsoreturnedbytheGET/_api/indexRESTAPImethodinasub-attributeselectivityEstimateforeachindexthatsupportsit.Thisattributewillbeomittedforindexesthatdonotprovideselectivityestimates.Ifprovided,theselectivityestimatewillbeanumericvaluebetween0and1.
Selectivityestimateswillalsobereportedintheresultofcollection.getIndexes()forallindexesthatsupportthis.Ifnoselectivityestimatecanbedeterminedforanindex,theattributeselectivityEstimatewillbeomittedhere,too.
Thewebinterfacealsoshowsselectivityestimatesforeachindexthatsupportsthis.
Currentlythefollowingindextypescanprovideselectivityestimates:
primaryindexedgeindexhashindex(uniqueandnon-unique)
Noselectivityestimateswillbeprovidedforindexeswhenrunninginclustermode.
AQLOptimizerimprovements
Sortremoval
WhatsNewin2.5
564
TheAQLoptimizerrule"use-index-for-sort"willnowremovesortsalsoincaseanon-sortedindex(e.g.ahashindex)isusedforonlyequalitylookupsandallsortattributesarecoveredbytheequalitylookupconditions.
Forexample,inthefollowingquerytheextrasortondoc.valuewillbeoptimizedawayprovidedthereisanindexondoc.value):
FORdocINcollection
FILTERdoc.value==1
SORTdoc.value
RETURNdoc
TheAQLoptimizerrule"use-index-for-sort"nowalsoremovessortincasethesortcriteriaexcludestheleft-mostindexattributes,buttheleft-mostindexattributesareusedbytheindexforequality-onlylookups.
Forexample,inthefollowingquerywithaskiplistindexonvalue1,value2,thesortcanbeoptimizedaway:
FORdocINcollection
FILTERdoc.value1==1
SORTdoc.value2
RETURNdoc
Constantattributepropagation
ThenewAQLoptimizerrulepropagate-constant-attributeswilllookforattributesthatareequality-comparedtoaconstantvalue,andwillpropagatethecomparisonvalueintootherequalitylookups.ThisrulewillonlylookinsideFILTERconditions,andinsertconstantvaluesfoundinFILTERs,too.
Forexample,therulewillinsert42insteadofi.valueinthesecondFILTERofthefollowingquery:
FORiINc1
FORjINc2
FILTERi.value==42
FILTERj.value==i.value
RETURN1
Interleavedprocessing
TheoptimizerwillnowinspectAQLdata-modificationqueriesanddetectifthequery'sdata-modificationpartcanruninlockstepwiththedataretrievalpartofthequery,orifthedataretrievalpartmustbeexecutedandcompletedfirstbeforethedata-modificationcanstart.
Executingbothdataretrievalanddata-modificationinlockstepallowsusingmuchsmallerbuffersforintermediateresults,reducingthememoryusageofqueries.Notallqueriesareeligibleforthisoptimization,andtheoptimizerwillonlyapplytheoptimizationwhenitcansafelydetectthatthedata-modificationpartofthequerywillnotmodifydatatobefoundbytheretrievalpart.
Queryexecutionstatistics
ThefilteredattributewasaddedtoAQLqueryexecutionstatistics.ThevalueofthisattributeindicateshowmanydocumentswerefilteredbyFilterNodesintheAQLquery.NotethatIndexRangeNodescanalsofilterdocumentsbyselectingonlytherequiredrangesfromtheindex.ThefilteredvaluewillnotincludetheworkdonebyIndexRangeNodes,butonlytheworkperformedbyFilterNodes.
Languageimprovements
DynamicattributenamesinAQLobjectliterals
ThischangeallowsusingarbitraryexpressionstoconstructattributenamesinobjectliteralsspecifiedinAQLqueries.Todisambiguateexpressionsandotherunquotedattributenames,dynamicattributenamesneedtobeenclosedinbrackets([and]).
Example:
WhatsNewin2.5
565
FORiIN1..100
RETURN{[CONCAT('value-of-',i)]:i}
AQLfunctions
ThefollowingAQLfunctionswereaddedin2.5:
MD5(value):generatesanMD5hashofvalueSHA1(value):generatesanSHA1hashofvalueRANDOM_TOKEN(length):generatesarandomstringvalueofthespecifiedlength
SimplifyFoxxusage
ThankstoouruserfeedbackwelearnedthatFoxxisapowerful,yetrathercomplicatedconcept.With2.5wemadeitlesscomplicatedwhilekeepingallitsstrength.Thatincludesarewriteofthedocumentationaswellassomecodechangesasfollows:
MovedFoxxapplicationstoadifferentfolder.
Until2.4foxxappswerestoredinthefollowingfolderstructure:<app-path>/databases/<dbname>/<appname>:<appversion>.Thiscausedsometroubleasappswherecachedbasedonnameandversionandupdatesdidnotapply.Alsothepathonfilesystemandtheapp'saccessURLhadnorelationtooneanother.NowthepathonfilesystemisidenticaltotheURL(excepttheappendedAPP):<app-path>/_db/<dbname>/<mointpoint>/APP
RewriteofFoxxrouting
TheroutingofFoxxhasbeenexposedtomajorinternalchangesweadjustedbecauseofuserfeedback.Thisallowsustosetthedevelopmentmodepermountpointwithouthavingtochangepathsandholdappsatseparatelocations.
FoxxDevelopmentmode
Thedevelopmentmodeuseduntil2.4isgone.Ithasbeenreplacedbyamuchmorematureversion.Thisincludesthedeprecationofthejavascript.dev-app-pathparameter,whichisuselesssince2.5.Insteadofhavingtwoseparateappdirectoriesforproductionanddevelopment,appsnowresideinoneplace,whichisusedforproductionaswellasfordevelopment.Appscanstillbeputintodevelopmentmode,changingtheirbehaviorcomparedtoproductionmode.Developmentmodeappsarestillrereadfromdiskateveryrequest,andstilltheyshipmoredebugoutput.
Thischangehasalsomadethestartupoptions--javascript.frontend-development-modeand--javascript.dev-app-pathobsolete.Theformeroptionwillnothaveanyeffectwhenset,andthelatteroptionisonlyreadandusedduringtheupgradeto2.5anddoesnothaveanyeffectslater.
Foxxinstallprocess
InstallingFoxxappshasbeenatwostepprocess:importthemintoArangoDBandmountthemataspecificmountpoint.Theseoperationshavebeenjoinedtogether.Youcaninstallanappatonemountpoint,that'sit.Nofetch,mount,unmount,purgecycleanymore.Thecommandshavebeensimplifiedtojust:
install:getyourFoxxappupandrunninguninstall:shutitdownanderaseitfromdisk
Foxxerroroutput
Until2.4theerrorsproducedbyFoxxwerenotoptimal.Often,theerrormessagewasjustunabletoparsemanifestandcontainedonlyaninternalstacktrace.In2.5wemademajorimprovementsthere,includingamuchmorefinegrainederroroutputthathelpsyoudebugyourFoxxapps.Theerrormessageprintedisnowmuchclosertoitssourceandshouldhelpyoutrackitdown.
AlsoweaddedthedefaulthandlersforunhandlederrorsinFoxxapps:
YouwillgetaniceinternalerrorpagewheneveryourFoxxappiscalledbutwasnotinstalledduetoanyerror
WhatsNewin2.5
566
Youwillgetapropererrormessagewhenhavinganuncaughterrorappearsinanyapproute
InproductionmodethemessagesabovewillNOTcontainanyinformationaboutyourFoxxinternalsandaresafetobeexposedtothirdpartyusers.Indevelopmentmodethemessagesabovewillcontainthestacktrace(ifavailable),makingiteasierforyourin-housedevstotrackdownerrorsintheapplication.
Foxxconsole
WeaddedaconsoleobjecttoFoxxapps.AllFoxxappsnowhaveaconsoleobjectimplementingthefamiliarConsoleAPIintheirglobalscope,whichcanbeusedtologdiagnosticmessagestothedatabase.Thisconsolealsoallowstoreadtheerroroutputofonespecificfoxx.
Foxxrequests
Weaddedorg/arangodb/requestmodule,whichprovidesasimpleAPIformakingHTTPrequeststoexternalservices.ThisisenablesFoxxtobedirectlypartofamicroservicearchitecture.
WhatsNewin2.5
567
IncompatiblechangesinArangoDB2.5ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB2.5,andadjustanyclientprogramsifnecessary.
Changedbehavior
V8
TheV8versionshippedwithArangoDBwasupgradedfrom3.29.59to3.31.74.1.ThisleadstoadditionalECMAScript6(ES6or"harmony")featuresbeingenabledbydefaultinArangoDB'sscriptingenvironment.
Apartfromthat,achangeintheinterpretationofcommand-lineoptionsbyV8mayaffectusers.ArangoDBpassesthevalueofthecommand-lineoption--javascript.v8-optionstoV8andleavesinterpretationofthecontentstoV8.Forexample,theArangoDBoption--javascript.v8-options="--harmony"couldbeusedtotellV8toenableitsharmonyfeatures.
InArangoDB2.4,thefollowingharmonyoptionsweremadeavailablebyV8:
--harmony_scoping(enableharmonyblockscoping)--harmony_modules(enableharmonymodules(impliesblockscoping))--harmony_proxies(enableharmonyproxies)--harmony_generators(enableharmonygenerators)--harmony_numeric_literals(enableharmonynumericliterals(0o77,0b11))--harmony_strings(enableharmonystring)--harmony_arrays(enableharmonyarrays)--harmony_arrow_functions(enableharmonyarrowfunctions)--harmony_classes(enableharmonyclasses)--harmony_object_literals(enableharmonyobjectliteralextensions)--harmony(enableallharmonyfeatures(exceptproxies))
Therewastheoption--harmony,whichturnedonalmostallharmonyfeatures.
InArangoDB2.5,V8providesthefollowingharmony-relatedoptions:
--harmony(enableallcompletedharmonyfeatures)--harmony_shipping(enableallshippedharmonyfeatures)--harmony_modules(enable"harmonymodules(impliesblockscoping)"(inprogress))--harmony_arrays(enable"harmonyarraymethods"(inprogress))--harmony_array_includes(enable"harmonyArray.prototype.includes"(inprogress))--harmony_regexps(enable"harmonyregularexpressionextensions"(inprogress))--harmony_arrow_functions(enable"harmonyarrowfunctions"(inprogress))--harmony_proxies(enable"harmonyproxies"(inprogress))--harmony_sloppy(enable"harmonyfeaturesinsloppymode"(inprogress))--harmony_unicode(enable"harmonyunicodeescapes"(inprogress))--harmony_tostring(enable"harmonytoString")--harmony_numeric_literals(enable"harmonynumericliterals")--harmony_strings(enable"harmonystringmethods")--harmony_scoping(enable"harmonyblockscoping")--harmony_classes(enable"harmonyclasses(impliesblockscoping&objectliteralextension)")--harmony_object_literals(enable"harmonyobjectliteralextensions")--harmony_templates(enable"harmonytemplateliterals")
Notethatthereareextraoptionsforbettercontrollingthededicatedfeatures,andespeciallythatthemeaningofthe--harmonyoptionhaschangedfromenablingallharmonyfeaturestoallcompletedharmonyfeatures!
Usersshouldadjustthevalueof--javascript.v8-optionsaccordingly.
PleasenotethatincompleteharmonyfeaturesaresubjecttochangeinfutureV8releases.
Incompatiblechangesin2.5
568
Sparseindexes
Hashindexesandskiplistindexescannowbecreatedinasparsevariant.Whennotexplicitlyset,thesparseattributedefaultstofalsefornewindexes.
Thiscausesachangeinbehaviorwhencreatingauniquehashindexwithoutspecifyingthesparseflag.Theuniquehashindexwillbecreatedinanon-sparsevariantinArangoDB2.5.
In2.4andbefore,uniquehashindexeswereimplicitlysparse,alwaysexcludingnullvaluesfromtheindex.Therewasnooptiontocontrolthisbehavior,andsparsitywasneithersupportedfornon-uniquehashindexesnorskiplistsin2.4.Thisimplicitsparsityofjustuniquehashindexeswasconsideredaninconsistency,andthereforethebehaviorwascleanedupin2.5.
Asof2.5,hashandskiplistindexeswillonlybecreatedsparseifsparsityisexplicitlyrequested.Thismayrequireachangeinindex-creatingclientcode,butonlyiftheclientcodecreatesuniquehashindexesandiftheyarestillintendedtobesparse.Inthiscase,theclientcodeshouldexplicitlysetthesparseflagtotruewhencreatingauniquehashindex.
Existinguniquehashindexesfrom2.4orbeforewillautomaticallybemigratedsotheyarestillsparseaftertheupgradeto2.5.Fortheseindexes,thesparseattributewillbepopulatedautomaticallywithavalueoftrue.
Geoindexesareimplicitlysparse,meaningdocumentswithouttheindexedlocationattributeorcontaininginvalidlocationcoordinatevalueswillbeexcludedfromtheindexautomatically.Thisisalsoachangewhencomparedtopre-2.5behavior,whendocumentswithmissingorinvalidcoordinatevaluesmayhavecausederrorsoninsertionwhenthegeoindex'uniqueflagwassetanditsignoreNullflagwasnot.
Thiswasconfusingandhasbeenrectifiedin2.5.ThemethodensureGeoConstraint()nowdoesthesameasensureGeoIndex().Furthermore,theattributesconstraint,unique,ignoreNullandsparseflagsarenowcompletelyignoredwhencreatinggeoindexes.ClientindexcreationcodethereforedoesnotneedtosettheignoreNullorconstraintattributeswhencreatingageoindex.
Thesameistrueforfulltextindexes.Thereisnoneedtospecifynon-uniquenessorsparsityforgeoorfulltextindexes.Theywillalwaysbenon-uniqueandsparse.
MovedFoxxapplicationstoadifferentfolder.
Until2.4foxxappswerestoredinthefollowingfolderstructure:<app-path>/databases/<dbname>/<appname>:<appversion>.Thiscausedsometroubleasappswherecachedbasedonnameandversionandupdatesdidnotapply.Alsothepathonfilesystemandtheapp'saccessURLhadnorelationtooneanother.NowthepathonfilesystemisidenticaltotheURL(excepttheappendedAPP):<app-path>/_db/<dbname>/<mointpoint>/APP
FoxxDevelopmentmode
Thedevelopmentmodeuseduntil2.4isgone.Ithasbeenreplacedbyamuchmorematureversion.Thisincludesthedeprecationofthejavascript.dev-app-pathparameter,whichisuselesssince2.5.Insteadofhavingtwoseparateappdirectoriesforproductionanddevelopment,appsnowresideinoneplace,whichisusedforproductionaswellasfordevelopment.Appscanstillbeputintodevelopmentmode,changingtheirbehaviorcomparedtoproductionmode.Developmentmodeappsarestillrereadfromdiskateveryrequest,andstilltheyshipmoredebugoutput.
Thischangehasalsomadethestartupoptions--javascript.frontend-development-modeand--javascript.dev-app-pathobsolete.Theformeroptionwillnothaveanyeffectwhenset,andthelatteroptionisonlyreadandusedduringtheupgradeto2.5anddoesnothaveanyeffectslater.
Foxxinstallprocess
InstallingFoxxappshasbeenatwostepprocess:importthemintoArangoDBandmountthemataspecificmountpoint.Theseoperationshavebeenjoinedtogether.Youcaninstallanappatonemountpoint,that'sit.Nofetch,mount,unmount,purgecycleanymore.Thecommandshavebeensimplifiedtojust:
install:getyourFoxxappupandrunninguninstall:shutitdownanderaseitfromdiskDeprecatedfeatures
Foxx:methodModel#toJSONSchema(id)isdeprecated,itwillraiseawarningifyouuseit.PleaseuseFoxx.toJSONSchema(id,model)
Incompatiblechangesin2.5
569
instead.
Removedfeatures
Startupswitch--javascript.frontend-development-mode:Itsmajorpurposewasinternaldevelopmentanyway.Nowthewebfrontendcanbesettodevelopmentmodesimilartoanyotherfoxxapp.Startupswitch--javascript.dev-app-path:WasusedforthedevelopmentmodeofFoxx.Thisisintegratedwiththenormalapp-pathnowandcanbetriggeredonapplevel.Thesecondapp-pathissuperfluous.Foxx:controller.collection:PleaseuseappContext.collectioninstead.Foxx:FoxxRepository.modelPrototype:PleaseuseFoxxRepository.modelinstead.Foxx:Model.extend({},{attributes:{}}):PleaseuseModel.extend({schema:{}})instead.Foxx:requestContext.bodyParam(paramName,description,Model):PleaseuserequestContext.bodyParam(paramName,options)instead.Foxx:requestContext.queryParam({type:string}):PleaseuserequestContext.queryParam({type:joi})instead.Foxx:requestContext.pathParam({type:string}):PleaseuserequestContext.pathParam({type:joi})instead.Graph:Themodulesorg/arangodb/graphandorg/arangodb/graph-blueprint:Pleaseusemoduleorg/arangodb/general-graphinstead.NOTE:Thisdoesnotmeanwedonotsupportblueprintsanymore.Generalgraphcoverseverythingthegraph--blueprintdid,plusmanymorefeatures.General-Graph:Inthemoduleorg/arangodb/general-graphthefunctions_undirectedRelationand_directedRelationarenolongeravailable.Bothfunctionshavebeenunifiedto_relation.
Incompatiblechangesin2.5
570
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.4.ArangoDB2.4alsocontainsseveralbugfixesthatarenotlistedhere.Foralistofbugfixes,pleaseconsulttheCHANGELOG.
V8versionupgrade
Thebuilt-inversionofV8hasbeenupgradedfrom3.16.14to3.29.59.ThisactivatesseveralES6(alsodubbedHarmonyorES.next)featuresinArangoDB,bothintheArangoShellandtheArangoDBserver.Theycanbeusedforscriptingandinserver-sideactionssuchasFoxxroutes,traversalsetc.
ThefollowingES6featuresareavailableinArangoDB2.4bydefault:
iteratorstheofoperatorsymbolspredefinedcollectionstypes(Map,Setetc.)typedarrays
ManyotherES6featuresaredisabledbydefault,butcanbemadeavailablebystartingarangodorarangoshwiththeappropriateoptions:
arrowfunctionsproxiesgeneratorsString,Array,andNumberenhancementsconstantsenhancedobjectandnumericliterals
ToactivatealltheseES6featuresinarangodorarangosh,startitwiththefollowingoptions:
arangosh--javascript.v8-options="--harmony--harmony_generators"
MoredetailsontheavailableES6featurescanbefoundinthisblog.
FoxxGeneratorArangoDB2.4isshippedwithFoxxGenerator,aframeworkforbuildingstandardizedHypermediaAPIseasily.ThegeneratedAPIscanbeconsumedwithclienttoolsthatunderstandSiren.
HypermediaisthesimpleideathatourHTTPAPIsshouldhavelinksbetweentheirendpointsinthesamewaythatourwebsiteshavelinksbetweenthem.FoxxGeneratorisbasedontheideathatyoucanrepresentanAPIasastatechart:Everyendpointisastateandthelinksarethetransitionsbetweenthem.Usingyourdescriptionofstatesandtransitions,itcanthencreateanAPIforyou.
TheFoxxGeneratorcancreateAPIsbasedonasemanticdescriptionofentitiesandtransitions.AblogseriesontheusecasesandhowtousetheFoxxgeneratorishere:
part1part2part3
AcookbookrecipeforgettingstartedwithFoxxGeneratorishere.
AQLimprovements
Optimizerimprovements
WhatsNewin2.4
571
TheAQLoptimizerhasbeenenhancedtouseofindexesinqueriesinseveraladditionalcases.FilterscontainingtheINoperatorcannowmakeuseofindexes,andmultipleOR-orAND-combinedfilterconditionscannowalsouseindexesifthefilterconditionsrefertothesameindexedattribute.
Hereareafewexamplesofqueriesthatcannowuseindexesbutcouldn'tbefore:
FORdocINcollection
FILTERdoc.indexedAttribute==1||doc.indexedAttribute>99
RETURNdoc
FORdocINcollection
FILTERdoc.indexedAttributeIN[3,42]||doc.indexedAttribute>99
RETURNdoc
FORdocINcollection
FILTER(doc.indexedAttribute>2&&doc.indexedAttribute<10)||
(doc.indexedAttribute>23&&doc.indexedAttribute<42)
RETURNdoc
Additionally,theoptimizerruleremove-filter-covered-by-indexhasbeenadded.ThisruleremovesFilterNodesandCalculationNodesfromanexecutionplanifthefilterconditionisalreadycoveredbyapreviousIndexRangeNode.Removingthefilter'sCalculationNodeandtheFilterNodeitselfwillspeedupqueryexecutionbecausethequeryrequireslesscomputation.
Furthermore,thenewoptimizerruleremove-sort-randwillremoveaSORTRAND()statementandmovetherandomiterationintotheappropriateEnumerateCollectionNode.Thisisusuallymoreefficientthanindividuallyenumeratingandsorting.
Data-modificationqueriesreturningdocuments
INSERT,REMOVE,UPDATEorREPLACEqueriesnowcanoptionallyreturnthedocumentsinserted,removed,updated,orreplaced.Thisishelpfulfortrackingtheauto-generatedattributes(e.g._key,_rev)createdbyanINSERTandinalotofothersituations.
Inordertoreturndocumentsfromadata-modificationquery,thestatementmustimmediatelybeimmediatelyfollowedbyaLETstatementthatassignseitherthepseudo-valueNEWorOLDtoavariable.ThisLETstatementmustbefollowedbyaRETURNstatementthatreturnsthevariableintroducedbyLET:
FORiIN1..100
INSERT{value:i}INtestLETinserted=NEWRETURNinserted
FORuINusers
FILTERu.status=='deleted'
REMOVEuINusersLETremoved=OLDRETURNremoved
FORuINusers
FILTERu.status=='notactive'
UPDATEuWITH{status:'inactive'}INusersLETupdated=NEWRETURNupdated
NEWreferstotheinsertedormodifieddocumentrevision,andOLDreferstothedocumentrevisionbeforeupdateorremoval.INSERTstatementscanonlyrefertotheNEWpseudo-value,andREMOVEoperationsonlytoOLD.UPDATEandREPLACEcanrefertoeither.
Inallcasesthefulldocumentswillbereturnedwithalltheirattributes,includingthepotentiallyauto-generatedattributessuchas_id,_key,or_revandtheattributesnotspecifiedintheupdateexpressionofapartialupdate.
Languageimprovements
COUNTclause
AnoptionalCOUNTclausewasaddedtotheCOLLECTstatement.TheCOUNTclauseallowsformoreefficientcountingofvalues.
InpreviousversionsofArangoDBonehadtowritethefollowingtocountdocuments:
RETURNLENGTH(
FORdocINcollection
FILTER...somecondition...
RETURNdoc
)
WhatsNewin2.4
572
WiththeCOUNTclause,thequerycanbemodifiedto
FORdocINcollection
FILTER...somecondition...
COLLECTWITHCOUNTINTOlength
RETURNlength
ThelatterquerywillbemuchmoreefficientbecauseitwillnotproduceanyintermediateresultswithneedtobeshippedfromasubqueryintotheLENGTHfunction.
TheCOUNTclausecanalsobeusedtocountthenumberofitemsineachgroup:
FORdocINcollection
FILTER...somecondition...
COLLECTgroup=doc.groupWITHCOUNTINTOlength
return{group:group,length:length}
COLLECTmodifications
InArangoDB2.4,COLLECToperationscanbemademoreefficientifonlyasmallfragmentofthegroupvaluesisneededlater.Forthesecases,COLLECTprovidesanoptionalconversionexpressionfortheINTOclause.Thisexpressioncontrolsthevaluethatisinsertedintothearrayofgroupvalues.Itcanbeusedforprojections.
ThefollowingqueryonlycopiesthedateRegisteredattributeofeachdocumentintothegroups,potentiallysavingalotofmemoryandcomputationtimecomparedtocopyingdoccompletely:
FORdocINcollection
FILTER...somecondition...
COLLECTgroup=doc.groupINTOdates=doc.dateRegistered
return{group:group,maxDate:MAX(dates)}
Comparethistothefollowingvariantofthequery,whichwastheonlywaytoachievethesameresultinpreviousversionsofArangoDB:
FORdocINcollection
FILTER...somecondition...
COLLECTgroup=doc.groupINTOdates
return{group:group,maxDate:MAX(dates[*].doc.dateRegistered)}
Theabovequerywillneedtocopythefulldocattributeintothelengthsvariable,whereasthenewvariantwillonlycopythedateRegisteredattributeofeachdoc.
Subquerysyntax
InpreviousversionsofArangoDB,subqueriesrequiredextraparenthesesaroundthem,andthiscausedconfusionwhensubquerieswereusedasfunctionparameters.Forexample,thefollowingquerydidnotwork:
LETvalues=LENGTH(
FORdocINcollectionRETURNdoc
)
buthadtobewrittenasfollows:
LETvalues=LENGTH((
FORdocINcollectionRETURNdoc
))
Thiswasunintuitiveandisfixedinversion2.4sothatbothvariantsofthequeryareacceptedandproducethesameresult.
Webinterface
WhatsNewin2.4
573
TheApplicationstabforFoxxapplicationsinthewebinterfacehasgotacompleteredesign.
ItwillnowonlyshowapplicationsthatarecurrentlyrunninginArangoDB.Foraselectedapplication,anewdetailedviewhasbeencreated.Thisviewprovidesabetteroverviewoftheapp,e.g.:
authorlicenseversioncontributorsdownloadlinksAPIdocumentation
InstallinganewFoxxapplicationontheserverismadeeasyusingthenewAddapplicationbutton.TheAddapplicationdialogprovidesallthefeaturesalreadyavailableinthefoxx-managerconsoleapplicationplussomemore:
installaFoxxapplicationfromGithubinstallaFoxxapplicationfromazipfileinstallaFoxxapplicationfromArangoDB'sapplicationstorecreateanewFoxxapplicationfromscratch:thisfeatureusesageneratortocreateaFoxxapplicationwithpre-definedCRUDmethodsforagivenlistofcollections.ThegeneratedFoxxappcaneitherbedownloadedasazipfileorbeinstalledontheserver.StartingwithanewFoxxapphasneverbeeneasier.
Miscellaneousimprovements
Defaultendpointis127.0.0.1
ThedefaultendpointfortheArangoDBserverhasbeenchangedfrom0.0.0.0to127.0.0.1.ThiswillmakenewArangoDBinstallationsunaccessiblefromclientsotherthanlocalhostunlesstheconfigurationischanged.Thisisasecurityprecautionmeasurethathasbeenrequestedasafeaturealotoftimes.
Ifyouarethedevelopmentoption--enable-relative,theendpointwillstillbe0.0.0.0.
Systemcollectionsinreplication
Bydefault,systemcollectionsarenowincludedinreplicationandallreplicationAPIreturnvalues.Thiswillleadtouseraccountsandcredentialsdatabeingreplicatedfrommastertoslaveservers.Thismayoverwriteslave-specificdatabaseusers.
Ifthisisundesired,the_userscollectioncanbeexcludedfromreplicationeasilybysettingtheincludeSystemattributetofalseinthefollowingcommands:
replication.sync({includeSystem:false});replication.applier.properties({includeSystem:false});
Thiswillexcludeallsystemcollections(including_aqlfunctions,_graphsetc.)fromtheinitialsynchronizationandthecontinuousreplication.
Ifthisisalsoundesired,itisalsopossibletospecifyalistofcollectionstoexcludefromtheinitialsynchronizationandthecontinuousreplicationusingtherestrictCollectionsattribute,e.g.:
require("org/arangodb/replication").applier.properties({
includeSystem:true,
restrictType:"exclude",
restrictCollections:["_users","_graphs","foo"]
});
WhatsNewin2.4
574
IncompatiblechangesinArangoDB2.4ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB2.4,andadjustanyclientprogramsifnecessary.
Changedbehavior
V8upgrade
ThebundledV8versionhasbeenupgradedfrom3.16.14to3.29.59.
Thenewversionprovidesbettererrorchecking,whichcanleadtosubtlechangesintheexecutionofJavaScriptcode.
Thefollowingcode,thoughnonsense,runswithouterrorin2.3and2.4whenstrictmodeisnotenabled:
(function(){
a=true;
a.foo=1;
})();
Whenenablingstrictmode,thefunctionwillthrowanerrorin2.4butnotin2.3:
(function(){
"usestrict";
a=true;
a.foo=1;
})();
TypeError:Cannotassigntoreadonlyproperty'foo'oftrue
Thoughthisisachangeinbehavioritcanbeconsideredanimprovement.Thenewversionactuallyuncoversanerrorthatwentundetectedintheoldversion.
Errormessageshavealsochangedslightlyinthenewversion.ApplicationsthatrelyontheexacterrormessagesoftheJavaScriptenginemayneedtobeadjustedsotheylookfortheupdatederrormessages.
Defaultendpoint
Thedefaultendpointforarangodisnow127.0.0.1.
ThischangewillmodifytheIPaddressArangoDBlistensonto127.0.0.1bydefault.ThiswillmakenewArangoDBinstallationsunaccessiblefromclientsotherthanlocalhostunlesstheconfigurationischanged.Thisisasecurityfeature.
TomakeArangoDBaccessiblefromanyclient,changetheserver'sconfiguration(--server.endpoint)toeithertcp://0.0.0.0:8529ortheserver'spubliclyvisibleIPaddress.
Replication
SystemcollectionsarenowincludedinthereplicationandallreplicationAPIreturnvaluesbydefault.
Thiswillleadtouseraccountsandcredentialsdatabeingreplicatedfrommastertoslaveservers.Thismayoverwriteslave-specificdatabaseusers.
Thismaybeconsideredafeatureorananti-feature,soitisconfigurable.
Ifreplicationofsystemcollectionsisundesired,theycanbeexcludedfromreplicationbysettingtheincludeSystemattributetofalseinthefollowingcommands:
initialsynchronization:replication.sync({includeSystem:false})continuousreplication:replication.applier.properties({includeSystem:false})
Incompatiblechangesin2.4
575
Thiswillexcludeallsystemcollections(including_aqlfunctions,_graphsetc.)fromtheinitialsynchronizationandthecontinuousreplication.
Ifthisisalsoundesired,itisalsopossibletospecifyalistofcollectionstoexcludefromtheinitialsynchronizationandthecontinuousreplicationusingtherestrictCollectionsattribute,e.g.:
require("org/arangodb/replication").applier.properties({
includeSystem:true,
restrictType:"exclude",
restrictCollections:["_users","_graphs","foo"]
});
Theaboveexamplewillingeneralincludesystemcollections,butwillexcludethespecifiedthreecollectionsfromcontinuousreplication.
TheHTTPRESTAPImethodsforfetchingthereplicationinventoryandfordumpingcollectionsalsosupporttheincludeSystemcontrolflagviaaURLparameterofthesamename.
Buildprocesschanges
Severaloptionsfortheconfigurecommandhavebeenremovedin2.4.Theoptions
--enable-all-in-one-v8
--enable-all-in-one-icu
--enable-all-in-one-libev
--with-libev=DIR
--with-libev-lib=DIR
--with-v8=DIR
--with-v8-lib=DIR
--with-icu-config=FILE
arenotavailableanymorebecausethebuildprocesswillalwaysusethebundledversionsofthelibraries.
WhenbuildingArangoDBfromsourceinadirectorythatalreadycontainedapre-2.4version,itwillbenecessarytorunamakesupercleancommandonceandafullrebuildafterwards:
gitpull
makesuperclean
makesetup
./configure<optionsgohere>
make
MiscellaneouschangesAsaconsequenceofglobalrenaminginthecodebase,theoptionmergeArrayshasbeenrenamedtomergeObjects.ThisoptioncontrolswhetherJSONobjectswillbemergedonanupdateoperationoroverwritten.Thedefaulthasbeen,andstillis,tomerge.Notspecifyingtheparameterwillleadtoamerge,asithasbeenthebehaviorinArangoDBeversince.
ThisaffectstheHTTPRESTAPImethodPATCH/_api/document/collection/key.ItsoptionalURLparametermergeArraysfortheoptionhasbeenrenamedtomergeObjects.
TheAQLUPDATEstatementisalsoaffected,asitsoptionmergeArrayshasalsobeenrenamedtomergeObjects.The2.3query
UPDATEdocINcollectionWITH{...}INcollectionOPTIONS{mergeArrays:false}
shouldthusberewrittentothefollowingin2.4:
UPDATEdocINcollectionWITH{...}INcollectionOPTIONS{mergeObjects:false}
Deprecatedfeatures
Incompatiblechangesin2.4
576
ForFoxxControllerobjects,themethodcollection()isdeprecatedandwillberemovedinfutureversionofArangoDB.Usingthismethodwillissueawarning.PleaseuseapplicationContext.collection()instead.
ForFoxxRepositoryobjects,thepropertymodelPrototypeisnowdeprecated.Usingitwillissueawarning.PleaseuseFoxxRepository.modelinstead.
InFoxxController/RequestContext,callingmethodbodyParam()withthreeargumentsisdeprecated.Pleaseuse.bodyParam(paramName,options)instead.
InFoxxController/RequestContextcallingmethodqueryParam({type:string})isdeprecated.PleaseuserequestContext.queryParam({type:joi})instead.
InFoxxController/RequestContextcallingmethodpathParam({type:string})isdeprecated.PleaseuserequestContext.pathParam({type:joi})instead.
ForFoxxModel,callingModel.extend({},{attributes:{}})isdeprecated.PleaseuseModel.extend({schema:{}})instead.
Inmoduleorg/arangodb/general-graph,thefunctions_undirectedRelation()and_directedRelation()aredeprecatedandwillberemovedinafutureversionofArangoDB.Bothfunctionshavebeenunifiedto_relation().
Themodulesorg/arangodb/graphandorg/arangodb/graph-blueprintaredeprecated.Pleaseusemoduleorg/arangodb/general-graphinstead.
TheHTTPRESTAPI_api/graphandallitsmethodsaredeprecated.PleaseusethegeneralgraphAPI_api/gharialinstead.
Removedfeatures
Thefollowingreplication-relatedJavaScriptmethodsbecameobsoleteinArangoDB2.2andhavebeenremovedinArangoDB2.4:
require("org/arangodb/replication").logger.start()
require("org/arangodb/replication").logger.stop()
require("org/arangodb/replication").logger.properties()
TheRESTAPImethodsforthesefunctionshavealsobeenremovedinArangoDB2.4:
HTTPPUT/_api/replication/logger-startHTTPPUT/_api/replication/logger-stopHTTPGET/_api/replication/logger-configHTTPPUT/_api/replication/logger-config
Clientapplicationsthatcalloneofthesemethodsshouldbeadjustedbyremovingthecallstothesemethods.Thisshouldn'tbeproblematicasthesemethodshavebeenno-opssinceArangoDB2.2anyway.
Incompatiblechangesin2.4
577
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.3.ArangoDB2.3alsocontainsseveralbugfixesthatarenotlistedhere.
AQLimprovements
Frameworkimprovements
AQLqueriesarenowsentthroughaqueryoptimizerframeworkbeforeexecution.Thequeryoptimizerframeworkwillfirstconverttheinternalrepresentationofthequery,theabstractsyntaxtree,intoaninitialexecutionplan.
Theexecutionplanisthensendthroughoptimizerrulesthatmaydirectlymodifytheplaninplaceorcreateanewvariantoftheplan.Newplansmightagainbeoptimized,allowingtheoptimizertocarryoutseveraloptimizations.
Aftercreatingplans,theoptimizerwillestimatethecostsforeachplanandpicktheplanwiththelowestcost(termedtheoptimalplan)fortheactualqueryexecution.
Withtheexplain()methodofArangoStatementuserscancheckwhichexecutionplantheoptimizerpickorretrievealistofotherplansthatoptimizerdidnotchoose.Theplanwillrevealmanydetailsaboutwhichindexesareusedetc.explain()willalsoreturntheofoptimizerrulesappliedsouserscanvalidatewhetherornotaqueryallowsusingaspecificoptimization.
ExecutionofAQLquerieshasbeenrewritteninC++,allowingmanyqueriestoavoidtheconversionofdocumentsbetweenArangoDB'sinternallow-leveldatastructureandtheV8objectrepresentationformat.
Theframeworkforoptimizerrulesisnowalsogenerallycluster-aware,allowingspecificoptimizationsforqueriesthatruninacluster.Additionally,theoptimizerwasdesignedtobeextensibleinordertoaddmoreoptimizationsinthefuture.
Languageimprovements
Alternativeoperatorsyntax
ArangoDB2.3allowstousethefollowingalternativeformsforthelogicaloperators:
AND:logicalandOR:logicalorNOT:negation
Thisnewsyntaxisjustanalternativetotheoldsyntax,allowingeasiermigrationfromSQL.Theoldsyntaxisstillfullysupportedandwillbe:
&&:logicaland||:logicalor!:negation
NOTINoperator
AQLnowhasadedicatedNOTINoperator.
Previously,aNOTINwasonlyachievablebywritinganegatedINcondition:
FORiIN...FILTER!(iIN[23,42])...
InArangoDB2.3,thesameresultcannowalternativelybeachievedbywritingthemoreintuitivevariant:
FORiIN...FILTERiNOTIN[23,42]...
Improvementsofbuilt-infunctions
WhatsNewin2.3
578
ThefollowingAQLstringfunctionshavebeenadded:
LTRIM(value,characters):left-trimsastringvalueRTRIM(value,characters):right-trimsastringvalueFIND_FIRST(value,search,start,end):findsthefirstoccurrenceofasearchstringFIND_LAST(value,search,start,end):findsthelastoccurrenceofasearchstringSPLIT(value,separator,limit):splitsastringintoanarray,usingaseparatorSUBSTITUTE(value,search,replace,limit):replacescharactersorstringsinsideanother
ThefollowingotherAQLfunctionshavebeenadded:
VALUES(document):returnsthevaluesofanobjectasanarray(thisisthecounterparttothealreadyexistingATTRIBUTESfunction)ZIP(attributes,values):returnsanobjectconstructedfromattributesandvaluespassedinseparateparametersPERCENTILE(values,n,method):returnsthenthspercentileofthevaluesprovided,usingrankorinterpolationmethod
ThealreadyexistingfunctionsCONCATandCONCAT_SEPARATORnowsupportarrayarguments,e.g.:
/*"foobarbaz"*/
CONCAT(['foo','bar','baz'])
/*"foo,bar,baz"*/
CONCAT_SEPARATOR(",",['foo','bar','baz'])
AQLqueriesthrowlessexceptions
InpreviousversionsofArangoDB,AQLqueriesabortedwithanexceptioninmanysituationsandthrewaruntimeexception.Forexample,exceptionswerethrownwhentryingtofindavalueusingtheINoperatorinanon-arrayelement,whentryingtousenon-booleanvalueswiththelogicaloperands&&or||or!,whenusingnon-numericvaluesinarithmeticoperations,whenpassingwrongparametersintofunctionsetc.
ThefactthatmanyAQLoperatorscouldthrowexceptionsledtoalotofquestionsfromusers,andalotofmore-verbose-than-necessaryqueries.Forexample,thefollowingqueryfailedwhenthereweredocumentsthatdidnothaveatopicsattributeatall:
FORdocINmycollection
FILTERdoc.topicsIN["something","whatever"]
RETURNdoc
Thisforceduserstorewritethequeryasfollows:
FORdocINmycollection
FILTERIS_LIST(doc.topics)&&doc.topicsIN["something","whatever"]
RETURNdoc
InArangoDB2.3thishasbeenchangedtomakeAQLeasiertouse.Thechangeprovidesanextrabenefit,andthatisthatnon-throwingoperatorsallowthequeryoptimizertoperformmuchmoretransformationsinthequerywithoutchangingitsoverallresult.
Hereisasummaryofchanges:
whenanon-arrayvalueisusedontheright-handsideoftheINoperator,theresultwillbefalseinArangoDB2.3,andnoexceptionwillbethrown.thebooleanoperators&&and||donotthrowinArangoDB2.3ifanyoftheoperandsisnotabooleanvalue.Instead,theywillperformanimplicitcastofthevaluestobooleans.Theirresultwillbeasfollows:
lhs&&rhswillreturnlhsifitisfalseorwouldbefalsewhenconvertedintoaboolean.Iflhsistrueorwouldbetruewhenconvertedtoaboolean,rhswillbereturned.lhs||rhswillreturnlhsifitistrueorwouldbetruewhenconvertedintoaboolean.Iflhsisfalseorwouldbefalsewhenconvertedtoaboolean,rhswillbereturned.!valuewillreturnthenegatedvalueofvalueconvertedintoaboolean
thearithmeticoperators(+,-,*,/,%)canbeappliedtoanyvalueandwillnotthrowexceptionswhenappliedtonon-numericvalues.Instead,anyvalueusedintheseoperatorswillbecastedtoanumericvalueimplicitly.Ifnonumericresultcanbeproducedbyanarithmeticoperator,itwillreturnnullinArangoDB2.3.Thisisalsotruefordivisionbyzero.passingargumentsofinvalidtypesintoAQLfunctionsdoesnotthrowaruntimeexceptioninmostcases,butmayproduceruntime
WhatsNewin2.3
579
warnings.Built-inAQLfunctionsthatreceiveinvalidargumentswillthenreturnnull.
Performanceimprovements
Non-uniquehashindexes
Theperformanceofinsertionintonon-uniquehashindexeshasbeenimprovedsignificantly.Thisfixesperformanceproblemsincaseattributeswereindexesthatcontainedonlyveryfewdistinctvalues,orwhenmostofthedocumentsdidnotevencontaintheindexedattribute.Thisalsofixesproblemswhenloadingcollectionswithsuchindexes.
Theinsertiontimenowscaleslinearlywiththenumberofdocumentsregardlessofthecardinalityoftheindexedattribute.
Reverseiterationoverskiplistindexes
AQLqueriescannowuseasortedskiplistindexforreverseiteration.ThisallowsseveralqueriestorunfasterthaninpreviousversionsofArangoDB.
Forexample,thefollowingAQLquerycannowusetheindexondoc.value:
FORdocINmycollection
FILTERdoc.value>23
SORTdoc.valuesDESC
RETURNdoc
PreviousversionsofArangoDBdidnotusetheindexbecauseofthedescending(DESC)sort.
Additionally,thenewAQLoptimizercanuseanindexforsortingnoweveniftheAQLquerydoesnotcontainaFILTERstatement.ThisoptimizationwasnotavailableinpreviousversionsofArangoDB.
AddedbasicsupportforhandlingbinarydatainFoxx
BufferobjectscannowbeusedwhensettingtheresponsebodyofanyFoxxaction.ThisallowsFoxxactionstoreturnbinarydata.
RequestswithbinarypayloadcanbeprocessedinFoxxapplicationsbyusingthenewmethodres.rawBodyBuffer().ThiswillreturntheunparsedrequestbodyasaBufferobject.
Thereisnowalsothemethodreq.requestParts()availableinFoxxtoretrievetheindividualcomponentsofamultipartHTTPrequest.Thatcanbeusedforexampletoprocessfileuploads.
Additionally,theres.send()methodhasbeenaddedasaconveniencemethodforreturningstrings,JSONobjectsorBuffersfromaFoxxaction.Itprovidessomeauto-detectionbasedonitsparametervalue:
res.send("<p>someHTML</p>");//returnsanHTMLstring
res.send({success:true});//returnsaJSONobject
res.send(newBuffer("somebinarydata"));//returnsbinarydata
Theconveniencemethodres.sendFile()cannowbeusedtoreturnthecontentsofafilefromaFoxxaction.Theyfilemaycontainbinarydata:
res.sendFile(applicationContext.foxxFilename("image.png"));
Thefilesystemmethodsfs.write()andfs.readBuffer()canbeusedtoworkwithbinarydata,too:
fs.write()willperformanauto-detectionofitssecondparameter'svaluesoitworkswithBufferobjects:
fs.write(filename,"somedata");//savesastringvalueinfile
fs.write(filename,newBuffer("somebinarydata"));//saves(binary)contentsofabuffer
fs.readBuffer()hasbeenaddedasamethodtoreadthecontentsofanarbitraryfileintoaBufferobject.
WhatsNewin2.3
580
Webinterface
Batchdocumentremovalandmovefunctionalityhasbeenaddedtothewebinterface,makingiteasiertoworkwithmultipledocumentsatonce.Additionally,basicJSONimportandexporttoolshavebeenadded.
Command-lineoptionsadded
Thecommand-lineoption--javascript.v8-contextswasaddedtoarangodtoprovidebettercontroloverthenumberofV8contextscreatedinarangod.
Previously,thenumberofV8contextsarangodcreatedatstartupwasequaltothenumberofserverthreads(asspecifiedbyoption--server.threads).
InsomesituationsitmaybemoresensibletocreatedifferentamountsofthreadsandV8contexts.ThisisbecauseeachV8contextscreatedwillconsumememoryandrequiresCPUresourcesforperiodicgarbagecollection.Contrary,serverthreadsdonothavesuchhighmemoryorCPUfootprint.
Iftheoption--javascript.v8-contextsisnotspecified,thenumberofV8contextscreatedatstartupwillremainequaltothenumberofserverthreads.ThusnochangeinconfigurationisrequiredtokeepthesamebehaviorasinpreviousArangoDBversions.
Thecommand-lineoption--log.use-local-timewasaddedtoprintdatesandtimesinArangoDB'slogintheserver-localtimezoneinsteadofUTC.Ifitisnotset,thetimezonewilldefaulttoUTC.
Theoption--backslash-escapehasbeenaddedtoarangoimp.SpecifyingthisoptionwillusethebackslashastheescapecharacterforliteralquoteswhenparsingCSVfiles.Theescapecharacterforliteralquotesisstillthedoublequotecharacter.
Miscellaneousimprovements
ArangoDB'sbuilt-inHTTPservernowsupportsHTTPpipelining.
TheArangoShelltutorialfromthearangodb.comwebsiteisnowintegratedintotheArangoDBshell.
PowerfulFoxxEnhancementsWiththenewjobqueuefeatureyoucanrunasyncjobstocommunicatewithexternalservices,FoxxqueriesmakewritingcomplexAQLqueriesmucheasierandFoxxsessionswillhandletheauthenticationandsessionhassleforyou.
FoxxQueries
WritinglongAQLqueriesinJavaScriptcanquicklybecomeunwieldy.Asof2.3ArangoDBbundlestheArangoDBQueryBuildermodulethatprovidesaJavaScriptAPIforwritingcomplexAQLquerieswithoutstringconcatenation.Allbuilt-infunctionsthatacceptAQLstringsnowsupportquerybuilderinstancesdirectly.AdditionallyFoxxprovidesamethodFoxx.createQueryforcreatingparametrizedqueriesthatcanreturnFoxxmodelsorapplyarbitrarytransformationstothequeryresults.
FoxxSessions
ThesessionfunctionalityinFoxxhasbeencompletelyrewritten.TheoldactivateAuthenticationAPIisstillsupportedbutmaybedeprecatedinthefuture.ThenewactivateSessionsAPIsupportscookiesorconfigurableheaders,providesoptionalJSONWebTokenandcryptographicsigningsupportandusesthenewsessionsFoxxapp.
ArangoDB2.3providesFoxxappsforusermanagementandsaltedhash-basedauthenticationwhichcanbereplacedwithorsupplementedbyalternativeimplementations.Foranexampleappusingboththebuilt-inauthenticationandOAuth2seetheFoxxSessionsExampleapp.
FoxxQueues
FoxxnowprovidesasyncworkersviatheFoxxQueuesAPI.Jobsenqueuedinajobqueuewillbeexecutedasynchronouslyoutsideoftherequest/responsecycleofFoxxcontrollersandcanbeusedtocommunicatewithexternalservicesorperformtasksthattakealongtimetocompleteormayrequiremultipleattempts.
WhatsNewin2.3
581
Jobscanbescheduledinadvanceorsettobeexecutedimmediately,thenumberofretryattempts,theretrydelayaswellassuccessandfailurehandlerscanbedefinedforeachjobindividually.Jobtypesthatintegratevariousexternalservicesfortransactionale-mails,loggingandusertrackingcanbefoundintheFoxxappregistry.
Misc
TherequestandresponseobjectsinFoxxcontrollersnowprovidemethodsforreadingandwritingrawcookiesandsignedcookies.
MountedFoxxappswillnowbeloadedwhenarangodstartsratherthanatthefirstdatabaserequest.Thismayresultinslightlyslowerstartuptimes(butafasterresponseforthefirstrequest).
WhatsNewin2.3
582
IncompatiblechangesinArangoDB2.3ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB2.3,andadjustanyclientprogramsifnecessary.
Configurationfilechanges
Threadsandcontexts
Thenumberofserverthreadsspecifiedisnowtheminimumofthreadsstarted.Therearesituationinwhichthreadsarewaitingforresultsofdistributeddatabaseservers.Inthiscasethenumberofthreadsisdynamicallyincreased.
WithArangoDB2.3,thenumberofserverthreadscanbeconfiguredindependentlyofthenumberofV8contexts.Theconfigurationoption--javascript.v8-contextswasaddedtoarangodtoprovidebettercontroloverthenumberofV8contextscreatedinarangod.
Previously,thenumberofV8contextsarangodcreatedatstartupwasequaltothenumberofserverthreads(asspecifiedbyoption--server.threads).
InsomesituationsitmaybemoresensibletocreatedifferentamountsofthreadsandV8contexts.ThisisbecauseeachV8contextscreatedwillconsumememoryandrequiresCPUresourcesforperiodicgarbagecollection.Contrary,serverthreadsdonothavesuchhighmemoryorCPUfootprint.
Iftheoption--javascript.v8-contextsisnotspecified,thenumberofV8contextscreatedatstartupwillremainequaltothenumberofserverthreads.ThusnochangeinconfigurationisrequiredtokeepthesamebehaviorasinpreviousArangoDBversions.
Ifyouareusingthedefaultconfigfilesormergethemwithyourlocalconfigfiles,pleasereviewifthedefaultnumberofserverthreadsisokayinyourenvironment.AdditionallyyoushouldverifythatthenumberofV8contextscreated(asspecifiedinoption--javascript.v8-contexts)isokay.
Syslog
Thecommand-lineoption--log.syslogwasusedinpreviousversionsofArangoDBtoturnloggingtosyslogonoroff:whensettingtoanon-emptystring,syslogloggingwasturnedon,otherwiseturnedoff.Whensyslogloggingwasturnedon,loggingwasdonewiththeapplicationnamespecifiedin--log.application,whichdefaultedtotriagens.Therewasalsoacommand-lineoption--log.hostnamewhichcouldbesetbutdidnothaveanyeffect.
Thisbehaviorturnedouttobeunintuitiveandwaschangedin2.3asfollows:
thecommand-lineoption--log.syslogisdeprecatedanddoesnothaveanyeffectwhenstartingArangoDB.toturnonsyslogloggingin2.3,theoption--log.facilityhastobesettoanon-emptystring.ThevalueforfacilityisOS-dependent(possiblevaluescanbefoundin/usr/include/syslog.horthelike-usershouldbeavailableonmanysystems).thedefaultvaluefor--log.applicationhasbeenchangedfromtriagenstoarangod.thecommand-lineoption--log.hostnameisdeprecatedanddoesnothaveanyeffectwhenstartingArangoDB.Instead,thehostnamewillbesetbysyslogautomatically.whenloggingtosyslog,ArangoDBnowomitsthedatetimeprefixandtheprocessid,becausethey'llbeaddedbysyslogautomatically.
AQL
AQLqueriesthrowlessexceptions
ArangoDB2.3containsacompletelyrewrittenAQLqueryoptimizerandexecutionengine.ThismeansthatAQLquerieswillbeexecutedwithadifferentenginethaninArangoDB2.2andearlier.PartsofAQLqueriesmightbeexecutedindifferentorderthanbeforebecausetheAQLoptimizerhasmorefreedomtomovethingsaroundinaquery.
Incompatiblechangesin2.3
583
InpreviousversionsofArangoDB,AQLqueriesabortedwithanexceptioninmanysituationsandthrewaruntimeexception.ExceptionswerethrownwhentryingtofindavalueusingtheINoperatorinanon-arrayelement,whentryingtousenon-booleanvalueswiththelogicaloperands&&or||or!,whenusingnon-numericvaluesinarithmeticoperations,whenpassingwrongparametersintofunctionsetc.
InArangoDB2.3thishasbeenchangedinmanycasestomakeAQLmoreuser-friendlyandtoallowtheoptimizationtoperformmuchmorequeryoptimizations.
Hereisasummaryofchanges:
whenanon-arrayvalueisusedontheright-handsideoftheINoperator,theresultwillbefalseinArangoDB2.3,andnoexceptionwillbethrown.thebooleanoperators&&and||donotthrowinArangoDB2.3ifanyoftheoperandsisnotabooleanvalue.Instead,theywillperformanimplicitcastofthevaluestobooleans.Theirresultwillbeasfollows:
lhs&&rhswillreturnlhsifitisfalseorwouldbefalsewhenconvertedintoaboolean.Iflhsistrueorwouldbetruewhenconvertedtoaboolean,rhswillbereturned.lhs||rhswillreturnlhsifitistrueorwouldbetruewhenconvertedintoaboolean.Iflhsisfalseorwouldbefalsewhenconvertedtoaboolean,rhswillbereturned.!valuewillreturnthenegatedvalueofvalueconvertedintoaboolean
thearithmeticoperators(+,-,*,/,%)canbeappliedtoanyvalueandwillnotthrowexceptionswhenappliedtonon-numericvalues.Instead,anyvalueusedintheseoperatorswillbecastedtoanumericvalueimplicitly.Ifnonumericresultcanbeproducedbyanarithmeticoperator,itwillreturnnullinArangoDB2.3.Thisisalsotruefordivisionbyzero.passingargumentsofinvalidtypesintoAQLfunctionsdoesnotthrowaruntimeexceptioninmostcases,butmayproduceruntimewarnings.Built-inAQLfunctionsthatreceiveinvalidargumentswillthenreturnnull.
NestedFORloopexecutionorder
Thequeryoptimizerin2.3maypermutetheorderofnestedFORloopsinAQLqueries,providedthatexchangingtheloopswillnotalteraqueryresult.However,achangeintheorderofreturnedvaluesisallowedbecausenosortorderisguaranteedbyAQL(andwasnever)unlessanexplicitSORTstatementisusedinaquery.
ChangedreturnvaluesofArangoQueryCursor.getExtra()
ThereturnvalueofArangoQueryCursor.getExtra()hasbeenchangedinArangoDB2.3.Itnowcontainsastatsattributewithstatisticsaboutthequerypreviouslyexecuted.Italsocontainsawarningsattributewithwarningsthathappenedduringqueryexecution.Thereturnvaluestructurehasbeenunifiedin2.3forbothread-onlyanddata-modificationqueries.
Thereturnvaluelookslikethisforaread-onlyquery:
arangosh>stmt=db._createStatement("FORiINmycollectionRETURNi");stmt.execute().getExtra()
{
"stats":{
"writesExecuted":0,
"writesIgnored":0,
"scannedFull":2600,
"scannedIndex":0
},
"warnings":[]
}
Fordata-modificationqueries,ArangoDB2.3returnsaresultwiththesamestructure:
arangosh>stmt=db._createStatement("FORiINxxREMOVEiINxx");stmt.execute().getExtra()
{
"stats":{
"writesExecuted":2600,
"writesIgnored":0,
"scannedFull":2600,
"scannedIndex":0
},
"warnings":[]
}
Incompatiblechangesin2.3
584
InArangoDB2.2,thereturnvalueofArangoQueryCursor.getExtra()wasemptyforread-onlyqueriesandcontainedanattributeoperationswithtwosub-attributesfordata-modificationqueries:
arangosh>stmt=db._createStatement("FORiINmycollectionRETURNi");stmt.execute().getExtra()
{
}
arangosh>stmt=db._createStatement("FORiINmycollectionREMOVEiINmycollection");stmt.execute().getExtra()
{
"operations":{
"executed":2600,
"ignored":0
}
}
ChangedreturnvaluesinHTTPmethodPOST/_api/cursor
ThepreviouslymentionedchangealsoleadstothestatisticsbeingreturnedintheHTTPRESTAPImethodPOST/_api/cursor.Previously,thereturnvaluecontainedanoptionalextraattributethatwasfilledonlyfordata-modificationqueriesandinsomeothercasesasfollows:
{
"result":[],
"hasMore":false,
"extra":{
"operations":{
"executed":2600,
"ignored":0
}
}
}
WiththechangedresultstructureinArangoDB2.3,theextraattributeintheresultwilllooklikethis:
{
"result":[],
"hasMore":false,
"extra":{
"stats":{
"writesExecuted":2600,
"writesIgnored":0,
"scannedFull":0,
"scannedIndex":0
},
"warnings":[]
}
}
IfthequeryoptionfullCountisrequested,thefullCountresultvaluewillalsobereturnedinsidethestatsattributeoftheextraattribute,andnotdirectlyasanattributeinsidetheextraattributeasin2.2.NotethatafullCountwillonlybepresentinextra.statsifitwasrequestedasanoptionforthequery.
TheresultinArangoDB2.3willalsocontainawarningsattributewiththearrayofwarningsthathappenedduringqueryexecution.
ChangedreturnvaluesinArangoStatement.explain()
ThereturnvalueofArangoStatement.explain()haschangedsignificantlyinArangoDB2.3.Thenewreturnvaluestructureisnotcompatiblewiththestructurereturnedby2.2.
InArangoDB2.3,thefullexecutionplanforanAQLqueryisreturnedalongsideallappliedoptimizerrules,optimizationwarningsetc.Itisalsopossibletohavetheoptimizerreturnallexecutionplans.Thisrequiredanewdatastructure.
ClientprogramsthatuseArangoStatement.explain()ortheHTTPRESTAPImethodPOST/_api/explainmayneedtobeadjustedtousethenewreturnformat.
Incompatiblechangesin2.3
585
ThereturnvalueofArangoStatement.parse()hasbeenextendedinArangoDB2.3.Inadditiontotheexistingattributes,ArangoDB2.3willalsoreturnanastattributecontainingtheabstractsyntaxtreeofthestatement.Thisextraattributecansafelybeignoredbyclientprograms.
Variablesnotupdatableinqueries
PreviousversionsofArangoDBallowedthemodificationofvariablesinsideAQLqueries,e.g.
LETcounter=0
FORiIN1..10
LETcounter=counter+1
RETURNcounter
Whilethisisadmittedlyaconvenientfeature,thenewqueryoptimizerdesigndidnotallowtokeepit.Additionally,updatingvariablesinsideaquerywouldpreventalotofoptimizationstoqueriesthatwewouldliketheoptimizertomake.Additionally,updatingvariablesinqueriesthatrunondifferentnodesinaclusterwouldlikecausenon-deterministicbehaviorbecausequeriesarenotexecutedlinearly.
ChangedreturnvalueofTO_BOOL
TheAQLfunctionTO_BOOLnowalwaysreturnstrueifitsargumentisanarrayoranobject.InpreviousversionsofArangoDB,thefunctionreturnedfalseforemptyarraysorforobjectswithoutattributes.
ChangedreturnvalueofTO_NUMBER
TheAQLfunctionTO_NUMBERnowreturnsnullifitsargumentisanobjectoranarraywithmorethanonemember.InpreviousversionofArangoDB,thereturnvalueinthesecaseswas0.TO_NUMBERwillreturn0foremptyarray,andthenumericequivalentofthearraymember'svalueforarrayswithasinglemember.
NewAQLkeywords
ThefollowingkeywordshavebeenaddedtoAQLinArangoDB2.3:
NOTANDOR
UnquotedusageofthesekeywordsforattributenamesinAQLquerieswilllikelyfailinArangoDB2.3.Ifanysuchattributenameneedstobeusedinaquery,itshouldbeenclosedinbacktickstoindicatetheusageofaliteralattributename.
Removedfeatures
Bitarrayindexes
Bitarrayindexeswereonlyhalf-waydocumentedandintegratedinpreviousversionsofArangoDBsotheirbenefitwaslimited.ThesupportforbitarrayindexeshasthusbeenremovedinArangoDB2.3.Itisnotpossibletocreateindexesoftype"bitarray"withArangoDB2.3.
WhenacollectionisopenedthatcontainsabitarrayindexdefinitioncreatedwithapreviousversionofArangoDB,ArangoDBwillignoreitandlogthefollowingwarning:
indextype'bitarray'isnotsupportedinthisversionofArangoDBandisignored
FutureversionsofArangoDBmayautomaticallyremovesuchindexdefinitionssothewarningswilleventuallydisappear.
Otherremovedfeatures
TheHTTPRESTAPImethodatPOST/_admin/modules/flushhasbeenremoved.
Incompatiblechangesin2.3
586
Knownissues
InArangoDB2.3.0,AQLqueriescontainingfilterconditionswithanINexpressionwillnotyetuseanindex:
FORdocINcollectionFILTERdoc.indexedAttributeIN[...]RETURNdoc
FORdocINcollection
FILTERdoc.indexedAttributeIN[...]
RETURNdoc
We’recurrentlyworkingongettingtheINoptimizationsdone,andwillshipthemina2.3maintenancereleasesoon(e.g.2.3.1or2.3.2).
Incompatiblechangesin2.3
587
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.2.ArangoDB2.2alsocontainsseveralbugfixesthatarenotlistedhere.
AQLimprovements
DatamodificationAQLqueries
Uptoincludingversion2.1,AQLsupporteddataretrievaloperationsonly.StartingwithArangoDBversion2.2,AQLalsosupportsthefollowingdatamodificationoperations:
INSERT:insertnewdocumentsintoacollectionUPDATE:partiallyupdateexistingdocumentsinacollectionREPLACE:completelyreplaceexistingdocumentsinacollectionREMOVE:removeexistingdocumentsfromacollection
Data-modificationoperationsarenormallycombinedwithotherAQLstatementssuchasFORloopsandFILTERconditionstodeterminethesetofdocumentstooperateon.Forexample,thefollowingquerywillfindalldocumentsincollectionusersthatmatchaspecificconditionandsettheirstatusvariabletoinactive:
FORuINusers
FILTERu.status=='notactive'
UPDATEuWITH{status:'inactive'}INusers
Thefollowingquerycopiesalldocumentsfromcollectionusersintocollectionbackup:
FORuINusers
INSERTuINbackup
Andthisqueryremovesdocumentsfromcollectionbackup:
FORdocINbackup
FILTERdoc.lastModified<DATE_NOW()-3600
REMOVEdocINbackup
Formoreinformationondata-modificationqueries,pleaserefertoDatamodificationqueries.
Updatablevariables
Previously,thevalueofavariableassignedinanAQLquerywiththeLETkeywordwasnotupdatableinanAQLquery.Thispreventedstatementslikethefollowingfrombeingexecutable:
LETsum=0
FORvINvalues
SORTv.year
LETsum=sum+v.value
RETURN{year:v.year,value:v.value,sum:sum}
OtherAQLimprovements
addedAQLTRANSLATEfunction
Thisfunctioncanbeusedtoperformlookupsfromstaticobjects,e.g.
LETcountryNames={US:"UnitedStates",UK:"UnitedKingdom",FR:"France"}
RETURNTRANSLATE("FR",countryNames)
WhatsNewin2.2
588
LETlookup={foo:"foo-replacement",bar:"bar-replacement",baz:"baz-replacement"}
RETURNTRANSLATE("foobar",lookup,"notcontained!")
Write-aheadlogAllwriteoperationsinanArangoDBserverwillnowbeautomaticallyloggedintheserver'swrite-aheadlog.Thewrite-aheadlogisasetofappend-onlylogfiles,anditisusedincaseofacrashrecoveryandforreplication.
Datafromthewrite-aheadlogwilleventuallybemovedintothejournalsordatafilesofcollections,allowingtheservertoremoveolderwrite-aheadlogfiles.
Cross-collectiontransactionsinArangoDBshouldbenefitconsiderablybythischange,aslesswritesthaninpreviousversionsarerequiredtoensurethedataofmultiplecollectionsareatomicallyanddurablycommitted.Alldata-modifyingoperationsinsidetransactions(insert,update,remove)willwritetheiroperationsintothewrite-aheadlogdirectlynow.Inpreviousversions,suchoperationswerebuffereduntilthecommitorrollbackoccurred.TransactionswithmultipleoperationsshouldthereforerequirelessphysicalmemorythaninpreviousversionsofArangoDB.
Thedatainthewrite-aheadlogcanalsobeusedinthereplicationcontext.InpreviousversionsofArangoDB,replicatingfromamasterrequiredturningonaspecialreplicationloggeronthemaster.Thereplicationloggercausedanextrawriteoperationintothe_replicationsystemcollectionforeachactualwriteoperation.Thisextrawriteisnowsuperfluous.Instead,slavescanreaddirectlyfromthemaster'swrite-aheadlogtogetinformedaboutmostrecentdatachanges.Thisremovestheneedtostoredata-modificationoperationsinthe_replicationcollectionaltogether.
Fortheconfigurationofthewrite-aheadlog,pleaserefertoWrite-aheadlogoptions.
Theintroductionofthewrite-aheadlogalsoremovestheneedtoconfigureandstartthereplicationloggeronamaster.ThoughthereplicationloggerobjectisstillavailableinArangoDB2.2toensureAPIcompatibility,starting,stopping,orconfiguringitwillhavenoeffect.
PerformanceimprovementsRemovedsortingofattributenameswhenincollectionshaper
InpreviousversionsofArangoDB,addingadocumentwithpreviouslynot-usedattributenamescausedafullsortofallattributenamesusedinthecollection.Thesortingwasdonetoensurefastcomparisonsofattributenamesinsomerareedgecases,butitconsiderablysloweddowninsertsintocollectionswithmanydifferentorevenuniqueattributenames.
Specializedprimaryindeximplementationtoallowfasterhashtablerebuildingandreducelookupsindatafilesfortheactualvalueof_key.Thisalsoreducestheamountofrandommemoryaccessesforprimaryindexinserts.
Reclamationofindexmemorywhendeletinglastdocumentincollection
Deletingdocumentsfromacollectiondidnotleadtoindexsizesbeingreduced.Instead,theindexmemorywaskeptallocatedandre-usedlaterwhenacollectionwasrefilledwithnewdocuments.Now,indexmemoryofprimaryindexesandhashindexesisreclaimedinstantlywhenthelastdocumentinacollectionisremoved.
Preventbufferingoflongprintresultsinarangosh'sandarangod'sprintcommand
Thischangewillemitbufferedintermediateprintresultsanddiscardtheoutputbuffertoquicklydeliverprintresultstotheuser,andtopreventconstructingverylargebuffersforlargeresults.
Miscellaneousimprovements
Addedinsertmethodasanaliasforsave.Documentscannowbeinsertedintoacollectionusingeithermethod:
db.test.save({foo:"bar"});
db.test.insert({foo:"bar"});
Cleanupofoptionsfordata-modificationoperations
WhatsNewin2.2
589
Manyofthedata-modificationoperationshadsignatureswithmanyoptionalboolparameters,e.g.:
db.test.update("foo",{bar:"baz"},true,true,true)
db.test.replace("foo",{bar:"baz"},true,true)
db.test.remove("foo",true,true)
db.test.save({bar:"baz"},true)
Suchlongparameterlistswereunintuitiveandhardtousewhenonlyoneoftheoptionalparametersshouldhavebeenset.
TomaketheAPIsmoreusable,theoperationsnowunderstandthefollowingalternativesignature:
collection.update(key,update-document,options)
collection.replace(key,replacement-document,options)
collection.remove(key,options)
collection.save(document,options)
Examples:
db.test.update("foo",{bar:"baz"},{overwrite:true,keepNull:true,waitForSync:true})
db.test.replace("foo",{bar:"baz"},{overwrite:true,waitForSync:true})
db.test.remove("foo",{overwrite:true,waitForSync:true})
db.test.save({bar:"baz"},{waitForSync:true})
Added--overwriteoptiontoarangoimp
Thisallowsremovingalldocumentsinacollectionbeforeimportingintoitusingarangoimp.
Honorstartupoption--server.disable-statisticswhendecidingwhetherornottostartperiodicstatisticscollectionjobs
Previously,thestatisticscollectionjobswerestartedeveniftheserverwasstartedwiththe--server.disable-statisticsflagbeingsettotrue.Nowiftheoptionissettotrue,nostatisticswillbecollectedontheserver.
DisallowstoringofJavaScriptobjectsthatcontainJavaScriptnativeobjectsoftypeDate,Function,RegExporExternal,e.g.
db.test.save({foo:/bar/});
db.test.save({foo:newDate()});
Thiswillnowprint
Error:<data>cannotbeconvertedintoJSONshape:couldnotshapedocument
Previously,objectsofthesetypesweresilentlyconvertedintoanemptyobject(i.e.{})andnowarningwasissued.
Tostoresuchobjectsinacollection,explicitlyconvertthemintostringslikethis:
db.test.save({foo:String(/bar/)});
db.test.save({foo:String(newDate())});
Removedfeatures
MRubyintegrationforarangod
ArangoDBhadanexperimentalMRubyintegrationinsomeofthepublishbuilds.Thiswasn'tcontinuouslydeveloped,andsoithasbeenremovedinArangoDB2.2.
Thischangehasledtothefollowingstartupoptionsbeingsuperfluous:
--ruby.gc-interval
--ruby.action-directory
--ruby.modules-path
--ruby.startup-directory
WhatsNewin2.2
590
SpecifyingthesestartupoptionswilldonothinginArangoDB2.2,sousingtheseoptionsshouldbeavoidedfromnowonastheymightberemovedinafutureversionofArangoDB.
Removedstartupoptions
ThefollowingstartupoptionshavebeenremovedinArangoDB2.2.Specifyingthemintheserver'sconfigurationfilewillnotproduceanerrortomakemigrationeasier.Still,usageoftheseoptionsshouldbeavoidedastheywillnothaveanyeffectandmightfullyberemovedinafutureversionofArangoDB:
--database.remove-on-drop
--database.force-sync-properties
--random.no-seed
--ruby.gc-interval
--ruby.action-directory
--ruby.modules-path
--ruby.startup-directory
--server.disable-replication-logger
MultiCollectionGraphs
ArangoDBisamultimodeldatabasewithnativegraphsupport.Inversion2.2thefeaturesforgraphshavebeenimprovedbyintegrationofanewgraphmodule.Allgraphscreatedwiththeoldmoduleareautomaticallymigratedintothenewmodulebutcanstillbeusedbytheoldmodule.
Newgraphmodule
Uptoincludingversion2.1,ArangoDBofferedamoduleforgraphsandgraphoperations.Thismoduleallowedyoutouseexactlyoneedgecollectiontogetherwithonevertexcollectioninagraph.WithArangoDBversion2.2thisgraphmoduleisdeprecatedandanewgraphmoduleisoffered.Thisnewmoduleallowstocombineanarbitrarynumberofvertexcollectionsandedgecollectionsinthesamegraph.Foreachedgecollectionalistofcollectionscontainingsourceverticesandalistofcollectionscontainingtargetverticescanbedefined.IfanedgeisstoredArangoDBchecksifthisedgeisvalidinthiscollection.Furthermoreifavertexisremovedfromoneofthecollectionsallconnectededgeswillberemovedaswell,givingtheguaranteeofnolooseendsinthegraphs.Thelayoutofthegraphcanbemodifiedatruntimebyaddingorremovingcollectionsandchangingthedefinitionsforedgecollections.Alloperationsonthegraphlevelaretransactionalbydefault.
GraphsinAQL
MulticollectiongraphshavebeenaddedtoAQLaswell.Basicfunctionality(gettingvertices,edges,neighbors)canbeexecutedusingtheentiregraph.Alsomoreadvancedfeatureslikeshortestpathcalculations,characteristicfactorsofthegraphortraversalshavebeenintegratedintoAQL.Forthesefunctionsallgraphscreatedwiththegraphmodulecanbeused.
WhatsNewin2.2
591
FeaturesandImprovementsThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.1.ArangoDB2.1alsocontainsseveralbugfixesthatarenotlistedhere.
NewEdgesIndex
Theedgesindex(usedtostoreconnectionsbetweennodesinagraph)internallyusesanewdatastructure.Thisdatastructureimprovestheperformancewhenpopulatingtheedgeindex(i.e.whenloadinganedgecollection).Forlargegraphsloadingcanbe20timesfasterthanwithArangoDB2.0.
Additionally,thenewindexfixesperformanceproblemsthatoccurredwhenmanyduplicate_fromor_tovalueswerecontainedintheindex.Furthermore,thenewindexsupportsfasterremovalofedges.
Finally,whenloadinganexistingcollectionandbuildingtheedgesindexforthecollection,lessmemoryre-allocationswillbeperformed.
Overall,thisshouldconsiderablyspeeduploadingedgecollections.
Thenewindextypereplacestheoldedgesindextypeautomatically,withoutanychangesbeingrequiredbytheenduser.
TheAPIofthenewindexiscompatiblewiththeAPIoftheoldindex.Stillitispossiblethatthenewindexreturnsedgesinadifferentorderthantheoldindex.Thisisstillconsideredtobecompatiblebecausetheoldindexhadneverguaranteedanyresultordereither.
AQLImprovements
AQLoffersfunctionalitytoworkwithdates.DatesarenodatatypesoftheirowninAQL(neithertheyareinJSON,whichisoftenusedasaformattoshipdataintoandoutofArangoDB).Instead,datesinAQLareinternallyrepresentedbyeithernumbers(timestamps)orstrings.ThedatefunctionsinAQLprovidemechanismstoconvertfromanumerictimestamptoastringrepresentationandviceversa.
TherearetwodatefunctionsinAQLtocreatedatesforfurtheruse:
DATE_TIMESTAMP(date)CreatesaUTCtimestampvaluefromdate
DATE_TIMESTAMP(year,month,day,hour,minute,second,millisecond):Sameasbefore,butallowsspecifyingtheindividualdatecomponentsseparately.Allparametersafterdayareoptional.
DATE_ISO8601(date):ReturnsanISO8601datetimestringfromdate.ThedatetimestringwillalwaysuseUTCtime,indicatedbytheZatitsend.
DATE_ISO8601(year,month,day,hour,minute,second,millisecond):sameasbefore,butallowsspecifyingtheindividualdatecomponentsseparately.Allparametersafterdayareoptional.
Thesetwoabovedatefunctionsacceptthefollowinginputvalues:
numerictimestamps,indicatingthenumberofmillisecondselapsedsincetheUNIXepoch(i.e.January1st197000:00:00UTC).Anexampletimestampvalueis1399472349522,whichtranslatesto2014-05-07T14:19:09.522Z.
datetimestringsinformatsYYYY-MM-DDTHH:MM:SS.MMM,YYYY-MM-DDHH:MM:SS.MMM,orYYYY-MM-DD.Millisecondsarealwaysoptional.
Atimezonedifferencemayoptionallybeaddedattheendofthestring,withthehoursandminutesthatneedtobeaddedorsubtractedtothedatetimevalue.Forexample,2014-05-07T14:19:09+01:00canbeusedtospecifyaonehouroffset,and2014-05-07T14:19:09+07:30canbespecifiedforsevenandhalfhoursoffset.Negativeoffsetsarealsopossible.Alternativelytoanoffset,aZcanbeusedtoindicateUTC/Zulutime.
Anexamplevalueis2014-05-07T14:19:09.522ZmeaningMay7th2014,14:19:09and522milliseconds,UTC/Zulutime.Anotherexamplevaluewithouttimecomponentis2014-05-07Z.
Pleasenotethatifnotimezoneoffsetisspecifiedinadatestring,ArangoDBwillassumeUTCtimeautomatically.Thisisdonetoensureportabilityofqueriesacrossserverswithdifferenttimezonesettings,andbecausetimestampswillalwaysbeUTC-based.
WhatsNewin2.1
592
individualdatecomponentsasseparatefunctionarguments,inthefollowingorder:
yearmonthdayhourminutesecondmillisecond
Allcomponentsfollowingdayareoptionalandcanbeomitted.Notethatnotimezoneoffsetscanbespecifiedwhenusingseparatedatecomponents,andUTC/Zulutimewillbeused.
ThefollowingcallstoDATE_TIMESTAMPareequivalentandwillallreturn1399472349522:
DATE_TIMESTAMP("2014-05-07T14:19:09.522")
DATE_TIMESTAMP("2014-05-07T14:19:09.522Z")
DATE_TIMESTAMP("2014-05-0714:19:09.522")
DATE_TIMESTAMP("2014-05-0714:19:09.522Z")
DATE_TIMESTAMP(2014,5,7,14,19,9,522)
DATE_TIMESTAMP(1399472349522)
ThesameistrueforcallstoDATE_ISO8601thatalsoacceptsvariableinputformats:
DATE_ISO8601("2014-05-07T14:19:09.522Z")
DATE_ISO8601("2014-05-0714:19:09.522Z")
DATE_ISO8601(2014,5,7,14,19,9,522)
DATE_ISO8601(1399472349522)
Theabovefunctionsareallequivalentandwillreturn"2014-05-07T14:19:09.522Z".
ThefollowingdatefunctionscanbeusedwithdatescreatedbyDATE_TIMESTAMPandDATE_ISO8601:
DATE_DAYOFWEEK(date):Returnstheweekdaynumberofdate.Thereturnvalueshavethefollowingmeanings:
0:Sunday1:Monday2:Tuesday3:Wednesday4:Thursday5:Friday6:Saturday
DATE_YEAR(date):Returnstheyearpartofdateasanumber.
DATE_MONTH(date):Returnsthemonthpartofdateasanumber.
DATE_DAY(date):Returnsthedaypartofdateasanumber.
DATE_HOUR(date):Returnsthehourpartofdateasanumber.
DATE_MINUTE(date):Returnstheminutepartofdateasanumber.
DATE_SECOND(date):Returnsthesecondspartofdateasanumber.
DATE_MILLISECOND(date):Returnsthemillisecondspartofdateasanumber.
Thefollowingotherdatefunctionsarealsoavailable:
DATE_NOW():Returnsthecurrenttimeasatimestamp.
Notethatthisfunctionisevaluatedoneveryinvocationandmayreturndifferentvalueswheninvokedmultipletimesinthesamequery.
ThefollowingotherAQLfunctionshavebeenaddedinArangoDB2.1:
FLATTEN:thisfunctioncanturnanarrayofsub-arraysintoasingleflatarray.Allarrayelementsintheoriginalarraywillbeexpandedrecursivelyuptoaconfigurabledepth.Theexpandedvalueswillbeaddedtothesingleresultarray.
WhatsNewin2.1
593
Example:
FLATTEN([1,2,[3,4],5,[6,7],[8,[9,10]])
willexpandthesub-arraysonthefirstlevelandproduce:
[1,2,3,4,5,6,7,8,[9,10]]
Tofullyflattenthearray,themaximumdepthcanbespecified(e.g.withavalueof2):
FLATTEN([1,2,[3,4],5,[6,7],[8,[9,10]],2)
Thiswillfullyexpandthesub-arraysandproduce:
[1,2,3,4,5,6,7,8,9,10]
CURRENT_DATABASE:thisfunctionwillreturnthenameofthedatabasethecurrentqueryisexecutedin.
CURRENT_USER:thisfunctionreturnsthenameofthecurrentuserthatisexecutingthequery.Ifauthorizationisturnedofforthequeryisexecutedoutsideofarequestcontext,nouserispresentandthefunctionwillreturnnull.
ClusterDumpandRestore
Thedumpandrestoretools,arangodumpandarangorestore,cannowbeusedtodumpandrestorecollectionsinacluster.Additionally,acollectiondumpfromastandaloneArangoDBservercanbeimportedintoacluster,andviceversa.
WebInterfaceImprovements
Thewebinterfaceinversion2.1hasamorecompactdashboard.Itprovideschartswithtime-seriesforincomingrequests,HTTPtransfervolumeandsomeserverresourceusagefigures.
Additionallyitprovidestrendindicators(e.g.15minaverages)anddistributioncharts(akahistogram)forsomefigures.
FoxxImprovements
ToeasilyaccessafileinsidethedirectoryofaFoxxapplicationfromwithinFoxx,Foxx'sapplicationContextnowprovidesthefoxxFilename()function.Itcanbeusedtoassemblethefullfilenameofafileinsidetheapplication'sdirectory.TheapplicationContextcanbeaccessedasglobalvariablefromanymodulewithinaFoxxapplication.
ThefilenamecanbeusedinsideFoxxactionsorsetup/teardownscripts,e.g.topopulateaFoxxapplication'scollectionwithdata.
TherequirefunctionnowalsopreferslocalmoduleswhenusedfrominsideaFoxxapplication.ThisallowsputtingmodulesinsidetheFoxxapplicationdirectoryandrequiringthemeasily.Italsoallowsusingapplication-specificversionsoflibrariesthatarebundledwithArangoDB(suchasunderscore.js).
WindowsInstaller
TheWindowsinstallershippedwithArangoDBnowsupportsinstallationofArangoDBforthecurrentuserorallusers,withtherequiredprivileges.ItalsosupportstheinstallationofArangoDBasaservice.
Fixesfor32bitsystems
SeveralissueshavebeenfixedthatoccurredonlywhenusingArangoDBona32bitsoperatingsystem,specifically:
acrashinathirdpartycomponentusedtomanageclusterdata
WhatsNewin2.1
594
athirdpartylibrarythatfailedtoinitializeon32bitWindows,makingarangodandarangoshcrashimmediately.
overflowsofvaluesusedfornanosecond-precisiontimeouts:theseoverflowshaveledtoinvalidvaluesbeingpassedtosocketoperations,makingthemfailandre-trytoooften
Updateddrivers
SeveraldriversforArangoDBhavebeencheckedforcompatibilitywith2.1.Thecurrentlistofdriverswithcompatibilitynotescanbefoundonlinehere.
C++11usage
WehavemovedseveralfilesfromCtoC++,allowingmorecodereuseandreducingtheneedforshippingdatabetweenthetwo.WehavealsodecidedtorequireC++11supportforArangoDB,whichallowsustousesomeofthesimplifications,featuresandguaranteesthatthisstandardhasinstock.
ThatalsomeansacompilerwithC++11supportisrequiredtobuildArangoDBfromsource.ForinstanceGNUCCofatleastversion4.8.
MiscellaneousImprovementsCancelableasynchronousjobs:severalpotentiallylong-runningjobscannowbecanceledviaanexplicitcanceloperation.Thisallowsstoppinglong-runningqueries,traversalsorscriptswithoutshuttingdownthecompleteArangoDBprocess.Jobcancelationisprovidedforasynchronouslyexecutedjobsasisdescribedin@refHttpJobCancel.
Server-sideperiodictaskmanagement:anArangoDBservernowprovidesfunctionalitytoregisterandunregisterperiodictasks.Tasksareuser-definedJavaScriptactionsthatcanberunperiodicallyandautomatically,independentofanyHTTPrequests.
Thefollowingtaskmanagementfunctionsareprovided:
require("org/arangodb/tasks").register():registersaperiodictaskrequire("org/arangodb/tasks").unregister():unregistersandremovesaperiodictaskrequire("org/arangodb/tasks").get():retrievesaspecifictasksorallexistingtasks
Anexampletask(tobeexecutedevery15seconds)canberegisteredlikethis:
vartasks=require("org/arangodb/tasks");
tasks.register({
name:"thisisanexampletaskwithparameters",
period:15,
command:function(params){
vargreeting=params.greeting;
vardata=JSON.stringify(params.data);
require('console').log('%sfromparametertask:%s',greeting,data);
},
params:{greeting:"hi",data:"howareyou?"}
});
Pleaserefertothesection@refTasksformoredetails.
Thefiguresmethodofacollectionnowreturnsdataaboutthecollection'sindexmemoryconsumption.Thereturnedvalueindexes.sizewillcontainthetotalamountofmemoryacquiredbyallindexesofthecollection.Thisfigurecanbeusedtoassessthememoryimpactofindexes.
CapitalizedHTTPresponseheaders:fromversion2.1,ArangoDBwillreturncapitalizedHTTPheadersbydefault,e.g.Content-Lengthinsteadofcontent-length.ThoughtheHTTPspecificationstatesthatheadersfieldnamearecase-insensitive,severalolderclienttoolsrelyonaspecificcaseinHTTPresponseheaders.ThischangesmakeArangoDBabitmorecompatiblewiththose.
Simplifiedusageofdb._createStatement():toeasilyrunanAQLquery,themethoddb._createStatementnowallowspassingtheAQLqueryasastring.Previouslyitrequiredtheusertopassanobjectwithaqueryattribute(whichthencontainedthequerystring).
WhatsNewin2.1
595
ArangoDBnowsupportsbothversions:
db._createStatement(queryString);
db._createStatement({query:queryString});
WhatsNewin2.1
596
AppendixReferences:Briefoverviewsoverinterfacesandobjects
db:thedbobjectcollection:thecollectionobject
JavaScriptModules:Listofbuilt-inandsupportedJSmodulesDeprecated:FeaturesthatareconsideredobsoleteandmaygetremovedeventuallyErrorcodesandmeanings:ListofallpossibleerrorsthatcanbeencounteredGlossary:Disambiguationpage
Appendix
597
References
References
598
The"db"ObjectThedbobjectisavailableinarangoshbydefault,andcanalsobeimportedandusedinFoxxservices.
db.namereturnsacollectionobjectforthecollectionname.
Thefollowingmethodsexistsonthe_dbobject:
Database
db._createDatabase(name,options,users)db._databases()db._dropDatabase(name,options,users)db._useDatabase(name)
Indexes
db._index(index)db._dropIndex(index)
Properties
db._id()db._isSystem()db._name()db._path()db._version()
Collection
db._collection(name)db._create(name)db._drop(name)db._truncate(name)
AQL
db._createStatement(query)db._query(query)db._explain(query)
Document
db._document(object)db._exists(object)db._remove(selector)db._replace(selector,data)db._update(selector,data)
db
599
The"collection"ObjectThefollowingmethodsexistonthecollectionobject(returnedbydb.name):
Collection
collection.checksum()collection.count()collection.drop()collection.figures()collection.load()collection.properties()collection.revision()collection.rotate()collection.toArray()collection.truncate()collection.type()collection.unload()
Indexes
collection.dropIndex(index)collection.ensureIndex(description)collection.getIndexes(name)collection.index(index)
Document
collection.all()collection.any()collection.closedRange(attribute,left,right)collection.document(object)collection.documents(keys)collection.edges(vertex-id)collection.exists(object)collection.firstExample(example)collection.inEdges(vertex-id)collection.insert(data)collection.edges(vertices)collection.iterate(iterator,options)collection.outEdges(vertex-id)collection.queryByExample(example)collection.range(attribute,left,right)collection.remove(selector)collection.removeByKeys(keys)collection.rename()collection.replace(selector,data)collection.replaceByExample(example,data)collection.save(data)collection.update(selector,data)collection.updateByExample(example,data)
collection
600
JavaScriptModulesArangoDBusesaNode.jscompatiblemodulesystem.Youcanusethefunctionrequireinordertoloadamoduleorlibrary.Itreturnstheexportedvariablesandfunctionsofthemodule.
Theglobalvariablesglobal,process,console,Buffer,__filenameand__dirnameareavailablethroughoutArangoDBandFoxx.
Nodecompatibilitymodules
ArangoDBsupportsanumberofmodulesforcompatibilitywithNode.js,including:
assertimplementsbasicassertionandtestingfunctions.
bufferimplementsabinarydatatypeforJavaScript.
consoleisawellknownloggingfacilitytoalltheJavaScriptdevelopers.ArangoDBimplementsmostoftheConsoleAPI,withtheexceptionsofprofileandcount.
eventsimplementsaneventemitter.
fsprovidesafilesystemAPIforthemanipulationofpaths,directories,files,links,andtheconstructionoffilestreams.ArangoDBimplementsmostFilesystem/Afunctions.
moduleprovidesdirectaccesstothemodulesystem.
pathimplementsfunctionsdealingwithfilenamesandpaths.
punycodeimplementsconversionfunctionsforpunycodeencoding.
querystringprovidesutilitiesfordealingwithquerystrings.
streamprovidesastreaminginterface.
string_decoderimplementslogicfordecodingbuffersintostrings.
urlprovidesutilitiesforURLresolutionandparsing.
utilprovidesgeneralutilityfunctionslikeformatandinspect.
AdditionallyArangoDBprovidespartialimplementationsforthefollowingmodules:
net:onlyisIP,isIPv4andisIPv6.
process:onlyenvandcwd;stubsforargv,stdout.isTTY,stdout.write,nextTick.
timers:stubsforsetImmediate,setTimeout,setInterval,clearImmediate,clearTimeout,clearIntervalandref.
tty:onlyisatty(alwaysreturnsfalse).
vm:onlyrunInThisContext.
ThefollowingNode.jsmodulesarenotavailableatall:child_process,cluster,constants,crypto(butsee@arangodb/cryptobelow),dgram,dns,domain,http(butsee@arangodb/requestbelow),https,os,sys,tls,v8,zlib.
ArangoDBSpecificModulesTherearealargenumberofArangoDB-specificmodulesusingthe@arangodbnamespace,mostlyforinternalusebyArangoDBitself.Thefollowinghoweverarenoteworthy:
@arangodbprovidesdirectaccesstothedatabaseanditscollections.
@arangodb/cryptoprovidesvariouscryptographyfunctionsincludinghashingalgorithms.
JavaScriptModules
601
@arangodb/requestprovidesthefunctionalityformakingsynchronousHTTP/HTTPSrequests.
@arangodb/foxxisthenamespaceprovidingthevariousbuildingblocksoftheFoxxmicroserviceframework.
BundledNPMModules
ThefollowingNPMmodulesarepreinstalled:
aqbistheArangoDBQueryBuilderandcanbeusedtoconstructAQLquerieswithachainingJavaScriptAPI.
chaiisafull-featuredassertionlibraryforwritingJavaScripttests.
dedentisasimpleutilityfunctionforformattingmulti-linestrings.
error-stack-parserparsesstacktracesintoamoreusefulformat.
graphql-syncisanArangoDB-compatibleGraphQLserver/schemaimplementation.
highlight.jsisanHTMLsyntaxhighlighter.
i(inflect)isautilitylibraryforinflecting(e.g.pluralizing)words.
iconv-liteisautilitylibraryforconvertingbetweencharacterencodings
joiisavalidationlibrarythatissupportedthroughouttheFoxxframework.
js-yamlisaJavaScriptimplementationoftheYAMLdataformat(apartialsupersetofJSON).
lodashisautilitybeltforJavaScriptprovidingvarioususefulhelperfunctions.
minimatchisaglobmatcherformatchingwildcardsinfilepaths.
qsprovidesutilitiesfordealingwithquerystringsusingadifferentformatthanthequerystringmodule.
semverisautilitylibraryforhandlingsemverversionnumbers.
sinonisamockinglibraryforwritingteststubs,mocksandspies.
timezoneisalibraryforconvertingdatetimevaluesbetweenformatsandtimezones.
JavaScriptModules
602
ArangoDBModuleconstarangodb=require('@arangodb')
Note:ThismoduleshouldnotbeconfusedwiththearangojsJavaScriptdriverwhichcanbeusedtoaccessArangoDBfromoutsidethedatabase.AlthoughtheAPIssharesimilaritiesandthefunctionalityoverlaps,thetwoarenotcompatiblewitheachotherandcannotbeusedinterchangeably.
The dbobjectarangodb.db
Thedbobjectrepresentsthecurrentdatabaseandletsyouaccesscollectionsandrunqueries.Formoreinformationseethedbobjectreference.
Examples
constdb=require('@arangodb').db;
constthirteen=db._query('RETURN5+8').next();
The aqltemplatestringhandlerarangodb.aql
TheaqlfunctionisaJavaScripttemplatestringhandler.ItcanbeusedtowritecomplexAQLqueriesasmulti-linestringswithouthavingtoworryaboutbindVarsandthedistinctionbetweencollectionsandregularparameters.
TouseitjustprefixaJavaScripttemplatestring(theoneswithbackticksinsteadofquotes)withitsimportname(e.g.aql)andpassinvariableslikeyouwouldwitharegulartemplatestring.ThestringwillautomaticallybeconvertedintoanobjectwithqueryandbindVarsattributeswhichyoucanpassdirectlytodb._querytoexecute.Ifyoupassinacollectionitwillbeautomaticallyrecognizedasacollectionreferenceandhandledaccordingly.
TofindoutmoreaboutAQLseetheAQLdocumentation.
Examples
constaql=require('@arangodb').aql;
constfilterValue=23;
constmydata=db._collection('mydata');
constresult=db._query(aql`
FORdIN${mydata}
FILTERd.num>${filterValue}
RETURNd
`).toArray();
The errorsobjectarangodb.errors
ThisobjectprovidesusefulobjectsforeacherrorcodeArangoDBmightuseinArangoErrorerrors.ThisishelpfulwhentryingtocatchspecificerrorsraisedbyArangoDB,e.g.whentryingtoaccessadocumentthatdoesnotexist.EachobjecthasacodepropertycorrespondingtotheerrorNumfoundonArangoErrorerrors.
Foracompletelistoftheerrornamesandcodesyoumayencounterseetheappendixonerrorcodes.
Examples
consterrors=require('@arangodb').errors;
@arangodb
603
try{
someCollection.document('does-not-exist');
}catch(e){
if(e.isArangoError&&e.errorNum===errors.ERROR_ARANGO_DOCUMENT_NOT_FOUND.code){
thrownewError('Documentdoesnotexist');
}
thrownewError('Somethingwentwrong');
}
The timefunctionarangodb.time
Thisfunctionprovidesthecurrenttimeinsecondsasafloatingpointvaluewithmicrosecondprecisison.
ThisfunctioncanbeusedinsteadofDate.now()whenadditionalprecisionisneeded.
Examples
consttime=require('@arangodb').time;
conststart=time();
db._query(someVerySlowQuery);
console.log(`Elapsedtime:${time()-start}secs`);
@arangodb
604
ConsoleModuleglobal.console===require('console')
Note:Youdon'tneedtoloadthismoduledirectly.TheconsoleobjectisgloballydefinedthroughoutArangoDBandprovidesaccesstoallfunctionsinthismodule.
console.assert
console.assert(expression,format,argument1,...)
Teststhatanexpressionistrue.Ifnot,logsamessageandthrowsanexception.
Examples
console.assert(value==="abc","expected:value===abc,actual:",value);
console.debug
console.debug(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultasdebugmessage.Notethatdebugmessageswillonlybeloggediftheserverisstartedwithloglevelsdebugortrace.
Stringsubstitutionpatterns,whichcanbeusedinformat.
%%sstring%%d,%%iinteger%%ffloatingpointnumber%%oobjecthyperlink
Examples
console.debug("%s","thisisatest");
console.dir
console.dir(object)
Logsalistingofallpropertiesoftheobject.
Exampleusage:
console.dir(myObject);
console.errorconsole.error(format,argument1,...)
Formatstheargumentsaccordingto@FA{format}andlogstheresultaserrormessage.
Stringsubstitutionpatterns,whichcanbeusedinformat.
%%sstring%%d,%%iinteger%%ffloatingpointnumber%%oobjecthyperlink
Exampleusage:
console.error("error'%s':%s",type,message);
console
605
console.getline
console.getline()
Readsinalinefromtheconsoleandreturnsitasstring.
console.groupconsole.group(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultaslogmessage.Opensanestedblocktoindentallfuturemessagessent.CallgroupEndtoclosetheblock.Representationofblockisuptotheplatform,itcanbeaninteractiveblockorjustasetofindentedsubmessages.
Exampleusage:
console.group("userattributes");
console.log("name",user.name);
console.log("id",user.id);
console.groupEnd();
console.groupCollapsedconsole.groupCollapsed(format,argument1,...)
Sameasconsole.group.
console.groupEnd
console.groupEnd()
Closesthemostrecentlyopenedblockcreatedbyacalltogroup.
console.infoconsole.info(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultasinfomessage.
Stringsubstitutionpatterns,whichcanbeusedinformat.
%%sstring%%d,%%iinteger%%ffloatingpointnumber%%oobjecthyperlink
Exampleusage:
console.info("The%sjumpedover%dfences",animal,count);
console.log
console.log(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultaslogmessage.Thisisanaliasforconsole.info.
console.timeconsole.time(name)
Createsanewtimerunderthegivenname.CalltimeEndwiththesamenametostopthetimerandlogthetimeelapsed.
Exampleusage:
console.time("mytimer");
...
console
606
console.timeEnd("mytimer");//thiswillprinttheelapsedtime
console.timeEndconsole.timeEnd(name)
Stopsatimercreatedbyacalltotimeandlogsthetimeelapsed.
console.timeEnd
console.trace()
LogsastacktraceofJavaScriptexecutionatthepointwhereitiscalled.
console.warnconsole.warn(format,argument1,...)
Formatstheargumentsaccordingtoformatandlogstheresultaswarnmessage.
Stringsubstitutionpatterns,whichcanbeusedinformat.
%%sstring%%d,%%iinteger%%ffloatingpointnumber%%oobjecthyperlink
console
607
CryptoModuleconstcrypto=require('@arangodb/crypto')
Thecryptomoduleprovidesimplementationsofvarioushashingalgorithmsaswellascryptographyrelatedfunctions.
Nonces
Thesefunctionsdealwithcryptographicnonces.
createNonce
crypto.createNonce():string
Createsacryptographicnonce.
Returnsthecreatednonce.
checkAndMarkNonce
crypto.checkAndMarkNonce(nonce):void
Checksandmarksanonce.
Arguments
nonce:string
Thenoncetocheckandmark.
Returnsnothing.
Randomvalues
Thefollowingfunctionsdealwithgeneratingrandomvalues.
rand
crypto.rand():number
Generatesarandomintegerthatmaybepositive,negativeorevenzero.
Returnsthegeneratednumber.
genRandomAlphaNumbers
crypto.genRandomAlphaNumbers(length):string
Generatesastringofrandomalpabeticalcharactersanddigits.
Arguments
length:number
Thelengthofthestringtogenerate.
Returnsthegeneratedstring.
genRandomNumbers
crypto.genRandomNumbers(length):string
Generatesastringofrandomdigits.
crypto
608
Arguments
length:number
Thelengthofthestringtogenerate.
Returnsthegeneratedstring.
genRandomSalt
crypto.genRandomSalt(length):string
Generatesastringofrandom(printable)ASCIIcharacters.
Arguments
length:number
Thelengthofthestringtogenerate.
Returnsthegeneratedstring.
JSONWebTokens(JWT)
ThesemethodsimplementtheJSONWebTokenstandard.
jwtEncode
crypto.jwtEncode(key,message,algorithm):string
GeneratesaJSONWebTokenforthegivenmessage.
Arguments
key:string|null
Thesecretcryptographickeytobeusedtosignthemessageusingthegivenalgorithm.Notethatthisfunctionwillraiseanerrorifthekeyisomittedbutthealgorithmexpectsakey,andalsoifthealgorithmdoesnotexpectakeybutakeyisprovided(e.g.whenusing"none").
message:string
MessagetobeencodedasJWT.Notethatthemessagewillonlybebase64-encodedandsigned,notencrypted.Donotstoresensitiveinformationintokensunlesstheywillonlybehandledbytrustedparties.
algorithm:string
Nameofthealgorithmtouseforsigningthemessage,e.g."HS512".
ReturnstheJSONWebToken.
jwtDecode
crypto.jwtDecode(key,token,noVerify):string|null
Arguments
key:string|null
Thesecretcryptographickeythatwasusedtosignthemessageusingthealgorithmindicatedbythetoken.Notethatthisfunctionwillraiseanerrorifthekeyisomittedbutthealgorithmexpectsakey.
Ifthealgorithmdoesnotexpectakeybutakeyisprovided,thetokenwillfailtoverify.
token:string
Thetokentodecode.
crypto
609
Notethatthefunctionwillraiseanerrorifthetokenismalformed(e.g.doesnothaveexactlythreesegments).
noVerify:boolean(Default:false)
Whetherverificationshouldbeskipped.Ifthisissettotruethesignatureofthetokenwillnotbeverified.Otherwisethefunctionwillraiseanerrorifthesignaturecannotbeverifiedusingthegivenkey.
ReturnsthedecodedJSONmessageornullifnotokenisprovided.
jwtAlgorithms
AhelperobjectcontainingthesupportedJWTalgorithms.EachattributenamecorrespondstoaJWTalgandthevalueisanobjectwithsignandverifymethods.
jwtCanonicalAlgorithmName
crypto.jwtCanonicalAlgorithmName(name):string
AhelperfunctionthattranslatesaJWTalgvaluefoundinaJWTheaderintothecanonicalnameofthealgorithminjwtAlgorithms.Raisesanerrorifnoalgorithmwithamatchingnameisfound.
Arguments
name:string
Algorithmnametolookup.
Returnsthecanonicalnameforthealgorithm.
Hashingalgorithms
md5crypto.md5(message):string
HashesthegivenmessageusingtheMD5algorithm.
Arguments
message:string
Themessagetohash.
Returnsthecryptographichash.
sha1
crypto.sha1(message):string
HashesthegivenmessageusingtheSHA-1algorithm.
Arguments
message:string
Themessagetohash.
Returnsthecryptographichash.
sha224
crypto.sha224(message):string
HashesthegivenmessageusingtheSHA-224algorithm.
Arguments
crypto
610
message:string
Themessagetohash.
Returnsthecryptographichash.
sha256crypto.sha256(message):string
HashesthegivenmessageusingtheSHA-256algorithm.
Arguments
message:string
Themessagetohash.
Returnsthecryptographichash.
sha384crypto.sha384(message):string
HashesthegivenmessageusingtheSHA-384algorithm.
Arguments
message:string
Themessagetohash.
Returnsthecryptographichash.
sha512
crypto.sha512(message):string
HashesthegivenmessageusingtheSHA-512algorithm.
Arguments
message:string
Themessagetohash.
Returnsthecryptographichash.
Miscellaneous
constantEqualscrypto.constantEquals(str1,str2):boolean
Comparestwostrings.Thisfunctioniteratesovertheentirelengthofbothstringsandcanhelpmakingcertaintimingattacksharder.
Arguments
str1:string
Thefirststringtocompare.
str2:string
Thesecondstringtocompare.
Returnstrueifthestringsareequal,falseotherwise.
pbkdf2
crypto
611
crypto.pbkdf2(salt,password,iterations,keyLength):string
GeneratesaPBKDF2-HMAC-SHA1hashofthegivenpassword.
Arguments
salt:string
Thecryptographicsalttohashthepasswordwith.
password:string
Themessageorpasswordtohash.
iterations:number
Thenumberofiterations.Thisshouldbeaveryhighnumber.OWASPrecommended64000iterationsin2012andrecommendsdoublingthatnumbereverytwoyears.
WhenusingPBKDF2forpasswordhashesitisalsorecommendedtoaddarandomvalue(typicallybetween0and32000)tothatnumberthatisdifferentforeachuser.
keyLength:number
Thekeylength.
Returnsthecryptographichash.
hmac
crypto.hmac(key,message,algorithm):string
GeneratesanHMAChashofthegivenmessage.
Arguments
key:string
Thecryptographickeytousetohashthemessage.
message:string
Themessagetohash.
algorithm:string
Thenameofthealgorithmtouse.
Returnsthecryptographichash.
crypto
612
FilesystemModuleconstfs=require('fs')
TheimplementationtriestofollowtheCommonJSFilesystem/A/0specificationwherepossible.
WorkingDirectory
Thedirectoryfunctionsbelowshouldn'tusethecurrentworkingdirectoryoftheserverlike.or./test.Youwillnotbeabletotellwhethertheenvironmenttheserverisrunninginwillpermitdirectorylisting,readingorwritingoffiles.
YoushouldeitherbaseyourdirectorieswithgetTempPath(),orasafoxxserviceusethemodule.context.basePath.
SingleFileDirectoryManipulation
exists
checksifafileofanytypeordirectoryexistsfs.exists(path)
Returnstrueifafile(ofanytype)oradirectoryexistsatagivenpath.Ifthefileisabrokensymboliclink,returnsfalse.
isFile
testsifpathisafilefs.isFile(path)
Returnstrueifthepathpointstoafile.
isDirectory
testsifpathisadirectoryfs.isDirectory(path)
Returnstrueifthepathpointstoadirectory.
size
getsthesizeofafilefs.size(path)
Returnsthesizeofthefilespecifiedbypath.
mtime
getsthelastmodificationtimeofafilefs.mtime(filename)
Returnsthelastmodificationdateofthespecifiedfile.ThedateisreturnedasaUnixtimestamp(numberofsecondselapsedsinceJanuary11970).
pathSeparatorfs.pathSeparator
Ifyouwanttocombinetwopathsyoucanusefs.pathSeparatorinsteadof/or\.
join
fs.join(path,filename)
Thefunctionreturnsthecombinationofthepathandfilename,e.g.fs.join(Hello/World,foo.bar)wouldreturnHello/World/foo.bar.
getTempFile
fs
613
returnsthenamefora(new)temporaryfilefs.getTempFile(directory,createFile)
Returnsthenameforanewtemporaryfileindirectorydirectory.IfcreateFileistrue,anemptyfilewillbecreatedsonootherprocesscancreateafileofthesamename.
Note:Thedirectorydirectorymustexist.
getTempPath
returnsthetemporarydirectoryfs.getTempPath()
Returnstheabsolutepathofthetemporarydirectory
makeAbsolute
makesagivenpathabsolutefs.makeAbsolute(path)
Returnsthegivenstringifitisanabsolutepath,otherwiseanabsolutepathtothesamelocationisreturned.
chmod
setsfilepermissionsofspecifiedfiles(nonwindowsonly)fs.exists(path)
Returnstrueonsuccess.
list
returnsthedirectorylistingfs.list(path)
Thefunctionsreturnsthenamesofallthefilesinadirectory,inlexicallysortedorder.Throwsanexceptionifthedirectorycannotbetraversed(orpathisnotadirectory).
Note:thismeansthatlist("x")ofadirectorycontaining"a"and"b"wouldreturn["a","b"],not["x/a","x/b"].
listTree
returnsthedirectorytreefs.listTree(path)
Thefunctionreturnsanarraythatstartswiththegivenpath,andallofthepathsrelativetothegivenpath,discoveredbyadepthfirsttraversalofeverydirectoryinanyvisiteddirectory,reportingbutnottraversingsymboliclinkstodirectories.Thefirstpathisalways"" ,thepathrelativetoitself.
makeDirectory
createsadirectoryfs.makeDirectory(path)
Createsthedirectoryspecifiedbypath.
makeDirectoryRecursive
createsadirectoryfs.makeDirectoryRecursive(path)
Createsthedirectoryhierarchyspecifiedbypath.
remove
removesafilefs.remove(filename)
Removesthefilefilenameatthegivenpath.Throwsanexceptionifthepathcorrespondstoanythingthatisnotafileorasymboliclink.If"path"referstoasymboliclink,removesthesymboliclink.
removeDirectory
fs
614
removesanemptydirectoryfs.removeDirectory(path)
Removesadirectoryifitisempty.Throwsanexceptionifthepathisnotanemptydirectory.
removeDirectoryRecursive
removesadirectoryfs.removeDirectoryRecursive(path)
Removesadirectorywithallsubelements.Throwsanexceptionifthepathisnotadirectory.
FileIO
read
readsinafilefs.read(filename)
Readsinafileandreturnsthecontentasstring.PleasenotethatthefilecontentmustbeencodedinUTF-8.
read64
readsinafileasbase64fs.read64(filename)
Readsinafileandreturnsthecontentasstring.ThefilecontentisBase64encoded.
readBuffer
readsinafilefs.readBuffer(filename)
ReadsinafileandreturnsitscontentinaBufferobject.
readFileSyncfs.readFileSync(filename,encoding)
Readsthecontentsofthefilespecifiedinfilename.Ifencodingisspecified,thefilecontentswillbereturnedasastring.Supportedencodingsare:
utf8orutf-8ascii
base64
ucs2orucs-2utf16leorutf16behex
Ifnoencodingisspecified,thefilecontentswillbereturnedinaBufferobject.
write
fs.write(filename,content)
Writesthecontentintoafile.ContentcanbeastringoraBufferobject.Ifthefilealreadyexists,itistruncated.
writeFileSync
fs.writeFileSync(filename,content)
Thisisanaliasforfs.write(filename,content).
appendfs.append(filename,content)
Writesthecontentintoafile.ContentcanbeastringoraBufferobject.Ifthefilealreadyexists,thecontentisappendedattheend.
fs
615
RecursiveManipulation
copyRecursive
copiesadirectorystructurefs.copyRecursive(source,destination)
Copiessourcetodestination.Exceptionswillbethrownon:
Failuretocopythefilespecifyingadirectoryfordestinationwhensourceisafilespecifyingadirectoryassourceanddestination
CopyFile
copiesafileintoatargetfilefs.copyFile(source,destination)
Copiessourcetodestination.IfDestinationisadirectory,afileofthesamenamewillbecreatedinthatdirectory,elsethecopywillgetthespecifiedfilename.
move
renamesafilefs.move(source,destination)
Movessourcetodestination.Failuretomovethefile,orspecifyingadirectoryfordestinationwhensourceisafilewillthrowanexception.Likewise,specifyingadirectoryassourceanddestinationwillfail.
ZIP
unzipFile
unzipsafilefs.unzipFile(filename,outpath,skipPaths,overwrite,password)
Unzipsthezipfilespecifiedbyfilenameintothepathspecifiedbyoutpath.Overwritesanyexistingtargetfilesifoverwriteissettotrue.
Returnstrueifthefilewasunzippedsuccessfully.
zipFile
zipsafilefs.zipFile(filename,chdir,files,password)
Storesthefilesspecifiedbyfilesinthezipfilefilename.Ifthefilefilenamealreadyexists,anerroristhrown.Thelistofinputfilesfilesmustbegivenasalistofabsolutefilenames.Ifchdirisnotempty,thechdirprefixwillbestrippedfromthefilenameinthezipfile,sowhenitisunzippedfilenameswillberelative.Specifyingapasswordisoptional.
Returnstrueifthefilewaszippedsuccessfully.
fs
616
RequestModuleconstrequest=require('@arangodb/request')
TherequestmoduleprovidesthefunctionalityformakingHTTPrequests.
MakingHTTPrequests
HTTPmethodhelpers
InadditiontotherequestfunctionconvenienceshorthandsareavailableforeachHTTPmethodintheformof,i.e.:
request.head(url,options)
request.get(url,options)
request.post(url,options)
request.put(url,options)
request.delete(url,options)
request.patch(url,options)
Theseareequivalenttousingtherequestfunctiondirectly,i.e.:
request[method](url,options)
//isequivalentto
request({method,url,...options});
Forexample:
constrequest=require('@arangodb/request');
request.get('http://localhost',{headers:{'x-session-id':'keyboardcat'}});
//isequivalentto
request({
method:'get',
url:'http://localhost',
headers:{'x-session-id':'keyboardcat'}
});
Therequestfunction
TherequestfunctioncanbeusedtomakeHTTPrequests.
request(options)
PerformsanHTTPrequestandreturnsaResponseobject.
Parameter
Therequestfunctiontakesthefollowingoptions:
urloruri:thefully-qualifiedURLoraparsedURLfromurl.parse.qs(optional):objectcontainingquerystringvaluestobeappendedtotheURL.useQuerystring:iftrue,usequerystringmoduletohandlequerystrings,otherwiseuseqsmodule.Default:false.method(optional):HTTPmethod(case-insensitive).Default:"GET".headers(optional):HTTPheaders(case-insensitive).Default:{}.body(optional):requestbody.MustbeastringorBuffer,oraJSONserializablevalueifjsonistrue.json:iftrue,bodywillbeserializedtoaJSONstringandtheContent-Typeheaderwillbesetto"application/json".AdditionallytheresponsebodywillalsobeparsedasJSON(unlessencodingissettonull).Default:false.form(optional):whensettoastringorobjectandnobodyhasbeenset,bodywillbesettoaquerystringrepresentationofthatvalueandtheContent-Typeheaderwillbesetto"application/x-www-form-urlencoded".AlsoseeuseQuerystring.auth(optional):anobjectwiththepropertiesusernameandpasswordforHTTPBasicauthenticationorthepropertybearerfor
request
617
HTTPBearertokenauthentication.
sslProtocol(optional):whichtlsversionshouldbeusedtoconnecttotheurl.Thedefaultis4whichisTLS1.0.Seesslprotocolformoreopitions.followRedirect:whetherHTTP3xxredirectsshouldbefollowed.Default:true.maxRedirects:themaximumnumberofredirectstofollow.Default:10.encoding:encodingtobeusedfortheresponsebody.Ifsettonull,theresponsebodywillbereturnedasaBuffer.Default:"utf-8".timeout:numberofmillisecondstowaitforaresponsebeforeabortingtherequest.returnBodyOnError:whethertheresponsebodyshouldbereturnedevenwhentheserverresponseindicatesanerror.Default:true.
ThefunctionreturnsaResponseobjectwiththefollowingproperties:
rawBody:therawresponsebodyasaBuffer.body:theparsedresponsebody.Ifencodingwasnotsettonull,thisisastring.Ifadditionallyjsonwassettotrueandtheresponsebodyiswell-formedJSON,thisistheparsedJSONdata.headers:anobjectcontainingtheresponseheaders.OtherwisethisisidenticaltorawBody.statusCodeandstatus:theHTTPstatuscodeoftheresponse,e.g.404.message:theHTTPstatusmessageoftheresponse,e.g.NotFound.
Forms
Therequestmodulesupportsapplication/x-www-form-urlencoded(URLencoded)formuploads:
constrequest=require('@arangodb/request');
varres=request.post('http://service.example/upload',{form:{key:'value'}});
//or
varres=request.post({url:'http://service.example/upload',form:{key:'value'}});
//or
varres=request({
method:'post',
url:'http://service.example/upload',
form:{key:'value'}
});
Formdatawillbeencodedusingtheqsmodulebydefault.
Ifyouwanttousethequerystringmoduleinstead,simplyusetheuseQuerystringoption.
JSON
IfyouwanttosubmitJSON-serializablevaluesasrequestbodies,justsetthejsonoption:
constrequest=require('@arangodb/request');
varres=request.post('http://service.example/notify',{body:{key:'value'},json:true});
//or
varres=request.post({url:'http://service.example/notify',body:{key:'value'},json:true});
//or
varres=request({
method:'post',
url:'http://service.example/notify',
body:{key:'value'},
json:true
});
HTTPauthentication
TherequestmodulesupportsbothHTTPBasicauthentication.Justpassthecredentialsviatheauthoption:
constrequest=require('@arangodb/request');
varres=request.get(
request
618
'http://service.example/secret',
{auth:{username:'jcd',password:'bionicman'}}
);
//or
varres=request.get({
url:'http://service.example/secret',
auth:{username:'jcd',password:'bionicman'}
});
//or
varres=request({
method:'get',
url:'http://service.example/secret',
auth:{username:'jcd',password:'bionicman'}
});
AlternativelyyoucansupplythecredentialsviatheURL:
constrequest=require('@arangodb/request');
varusername='jcd';
varpassword='bionicman';
varres=request.get(
'http://'+
encodeURIComponent(username)+
':'+
encodeURIComponent(password)+
'@service.example/secret'
);
YoucanalsouseBearertokenauthentication:
constrequest=require('@arangodb/request');
varres=request.get(
'http://service.example/secret',
{auth:{bearer:'keyboardcat'}}
);
//or
varres=request.get({
url:'http://service.example/secret',
auth:{bearer:'keyboardcat'}
});
//or
varres=request({
method:'get',
url:'http://service.example/secret',
auth:{bearer:'keyboardcat'}
});
request
619
Module"actions"constactions=require('@arangodb/actions')
Theactionmoduleprovidestheinfrastructurefordefininglow-levelHTTPactions.
IfyouwanttodefineHTTPendpointsinArangoDByoushouldprobablyusetheFoxxmicroserviceframeworkinstead.
Basics
Errormessage
actions.getErrorMessage(code)
Returnstheerrormessageforanerrorcode.
StandardHTTPResultGeneratorsactions.defineHttp(options)
Definesanewaction.Theoptionsareasfollows:
options.url
TheURL,whichcanbeusedtoaccesstheaction.Thispathmightcontainslashes.Notethatthisactionwillalsobecalled,ifaurlisgivensuchthatoptions.urlisaprefixofthegivenurlandnolongerdefinitionmatches.
options.prefix
Iffalse,thenonlyusetheactionforexactmatches.Thedefaultistrue.
options.callback(request,response)
Therequestargumentcontainsadescriptionoftherequest.Arequestparameterfooisaccessibleasrequest.parametrs.foo.Arequestheaderbarisaccessibleasrequest.headers.bar.Assumethattheactionisdefinedfortheurl/foo/barandtherequesturlis/foo/bar/hugo/egon.Thenthesuffixparts["hugo","egon"]areavailibleinrequest.suffix.
Thecallbackmustdefinefilltheresponse.
response.responseCode:theresponsecoderesponse.contentType:thecontenttypeoftheresponseresponse.body:thebodyoftheresponse
YoucanusethefunctionsResultOkandResultErrortoeasilygeneratearesponse.
Resultok
actions.resultOk(req,res,code,result,headers)
Thefunctiondefinesaresponse.codeisthestatuscodetoreturn.resultistheresultobject,whichwillbereturnedasJSONobjectinthebody.headersisanarrayofheaderstoreturned.Thefunctionaddstheattributeerrorwithvaluefalseandcodewithvaluecodetotheresult.
Resultbad
actions.resultBad(req,res,error-code,msg,headers)
Thefunctiongeneratesanerrorresponse.
Resultnotfoundactions.resultNotFound(req,res,code,msg,headers)
actions
620
Thefunctiongeneratesanerrorresponse.
Resultunsupported
actions.resultUnsupported(req,res,headers)
Thefunctiongeneratesanerrorresponse.
Resulterror
actions.resultError(req,res,code,errorNum,errorMessage,headers,keyvals)*
Thefunctiongeneratesanerrorresponse.TheresponsebodyisanarraywithanattributeerrorMessagecontainingtheerrormessageerrorMessage,errorcontainingtrue,codecontainingcode,errorNumcontainingerrorNum,anderrorMessagecontainingtheerrormessageerrorMessage.keyvalsaremixedintotheresult.
ResultnotImplementedactions.resultNotImplemented(req,res,msg,headers)
Thefunctiongeneratesanerrorresponse.
Resultpermanentredirect
actions.resultPermanentRedirect(req,res,options,headers)
Thefunctiongeneratesaredirectresponse.
Resulttemporaryredirectactions.resultTemporaryRedirect(req,res,options,headers)
Thefunctiongeneratesaredirectresponse.
ArangoDBResultGenerators
Collectionnotfoundactions.collectionNotFound(req,res,collection,headers)
Thefunctiongeneratesanerrorresponse.
Indexnotfound
actions.indexNotFound(req,res,collection,index,headers)
Thefunctiongeneratesanerrorresponse.
Resultexceptionactions.resultException(req,res,err,headers,verbose)
Thefunctiongeneratesanerrorresponse.If@FA{verbose}issettotrueornotspecified(thedefault),thentheerrorstacktracewillbeincludedintheerrormessageifavailable.If@FA{verbose}isastringitwillbeprependedbeforetheerrormessageandthestacktracewillalsobeincluded.
actions
621
QueriesModuleconstqueries=require('@arangodb/aql/queries')
ThequerymoduleprovidestheinfrastructureforworkingwithcurrentlyrunningAQLqueriesviaarangosh.
Properties
queries.properties()Returnstheserverscurrentquerytrackingconfiguration;wechangetheslowquerythresholdtogetbetterresults:
arangosh>varqueries=require("@arangodb/aql/queries");
arangosh>queries.properties();
arangosh>queries.properties({slowQueryThreshold:1});
showexecutionresults
Currentlyrunningqueries
Wecreateataskthatspawnsqueries,sowehaveniceoutput.Sincethistaskusesresources,youmaywanttoincreaseperiod(andnotforgettoremoveit...afterwards):
arangosh>vartheQuery='FORsleepLoooongIN1..5LETsleepLoooonger=SLEEP(1000)RETURN
sleepLoooong';
arangosh>vartasks=require("@arangodb/tasks");
arangosh>tasks.register({
........>id:"mytask-1",
........>name:"thisisasampletasktospawnaslowaqlquery",
........>command:"require('@arangodb').db._query('"+theQuery+"');"
........>});
arangosh>queries.current();
showexecutionresultsThefunctionreturnsthecurrentlyrunningAQLqueriesasanarray.
SlowqueriesThefunctionreturnsthelastAQLqueriesthatexceededtheslowquerythresholdasanarray:
arangosh>queries.slow();
[]
ClearslowqueriesClearthelistofslowAQLqueries:
arangosh>queries.clearSlow();
{
"code":200
}
arangosh>queries.slow();
[]
queries
622
KillKillarunningAQLquery:
arangosh>varrunningQueries=queries.current().filter(function(query){
........>returnquery.query===theQuery;
........>});
arangosh>queries.kill(runningQueries[0].id);
{
"code":200
}
queries
623
Write-aheadlogconstwal=require('internal').wal
Thismoduleprovidesfunctionalityforadministeringthewrite-aheadlogs.Mostofthesefunctionsonlyreturnsensiblevalueswheninvokedwiththemmfilesenginebeingactive.
Configuration
retrievestheconfigurationofthewrite-aheadloginternal.wal.properties()
Retrievestheconfigurationofthewrite-aheadlog.TheresultisaJSONarraywiththefollowingattributes:
allowOversizeEntries:whetherornotoperationsthatarebiggerthanasinglelogfilecanbeexecutedandstoredlogfileSize:thesizeofeachwrite-aheadlogfilehistoricLogfiles:themaximumnumberofhistoriclogfilestokeepreserveLogfiles:themaximumnumberofreservelogfilesthatArangoDBallocatesinthebackgroundsyncInterval:theintervalforautomaticsynchronizationofnot-yetsynchronizedwrite-aheadlogdata(inmilliseconds)throttleWait:themaximumwaittimethatoperationswillwaitbeforetheygetabortedifcaseofwrite-throttling(inmilliseconds)throttleWhenPending:thenumberofunprocessedgarbage-collectionoperationsthat,whenreached,willactivatewrite-throttling.Avalueof0meansthatwrite-throttlingwillnotbetriggered.
Examples
arangosh>require("internal").wal.properties();
showexecutionresultsconfiguresthewrite-aheadloginternal.wal.properties(properties)
Configuresthebehaviorofthewrite-aheadlog.propertiesmustbeaJSONJSONobjectwiththefollowingattributes:
allowOversizeEntries:whetherornotoperationsthatarebiggerthanasinglelogfilecanbeexecutedandstoredlogfileSize:thesizeofeachwrite-aheadlogfilehistoricLogfiles:themaximumnumberofhistoriclogfilestokeepreserveLogfiles:themaximumnumberofreservelogfilesthatArangoDBallocatesinthebackgroundthrottleWait:themaximumwaittimethatoperationswillwaitbeforetheygetabortedifcaseofwrite-throttling(inmilliseconds)throttleWhenPending:thenumberofunprocessedgarbage-collectionoperationsthat,whenreached,willactivatewrite-throttling.Avalueof0meansthatwrite-throttlingwillnotbetriggered.
Specifyinganyoftheaboveattributesisoptional.Notspecifiedattributeswillbeignoredandtheconfigurationforthemwillnotbemodified.
Examples
arangosh>require("internal").wal.properties({
........>allowOverSizeEntries:true,
........>logfileSize:32*1024*1024});
showexecutionresults
Flushing
flushesthecurrentlyopenWALlogfileinternal.wal.flush(waitForSync,waitForCollector)
Flushesthewrite-aheadlog.Byflushingthecurrentlyactivewrite-aheadlogfile,thedatainitcanbetransferredtocollectionjournalsanddatafiles.Thisisusefultoensurethatalldataforacollectionispresentinthecollectionjournalsanddatafiles,forexample,whendumpingthedataofacollection.
Write-aheadlog
624
ThewaitForSyncoptiondetermineswhetherornottheoperationshouldblockuntilthenot-yetsynchronizeddatainthewrite-aheadlogwassynchronizedtodisk.
ThewaitForCollectoroperationcanbeusedtospecifythattheoperationshouldblockuntilthedataintheflushedloghasbeencollectedbythewrite-aheadloggarbagecollector.Notethatsettingthisoptiontotruemightblockforalongtimeiftherearelong-runningtransactionsandthewrite-aheadloggarbagecollectorcannotfinishgarbagecollection.
Examples
arangosh>require("internal").wal.flush();
Write-aheadlog
625
TaskManagementconsttasks=require('@arangodb/tasks')
Note:IfyouaretryingtoscheduletasksinFoxxyoushouldconsiderusingtheFoxxqueuesmoduleinstead,whichprovidesamorehigh-levelAPIthatalsopersiststasksacrossreboots.
IntroductiontoTaskManagementinArangoDB
ArangoDBcanexecuteuser-definedJavaScriptfunctionsasone-shotorperiodictasks.Thisfunctionalitycanbeusedtoimplementtimedorrecurringjobsinthedatabase.
TasksinArangoDBconsistofaJavaScriptsnippetorfunctionthatisexecutedwhenthetaskisscheduled.Ataskcanbeaone-shottask(meaningitisrunonceandnotrepeated)oraperiodictask(meaningthatitisre-scheduledaftereachexecution).Taskscanhaveoptionalparameters,whicharedefinedattasksetuptime.Theparametersspecifiedattasksetuptimewillbepassedasargumentstothetaskwheneveritgetsexecuted.PeriodicTaskshaveanexecutionfrequencythatneedstobespecifiedwhenthetaskissetup.One-shottaskshaveaconfigurabledelayafterwhichthey'llgetexecuted.
Taskswillbeexecutedontheservertheyhavebeensetupon.Taskswillnotbeshippedaroundinacluster.Ataskwillbeexecutedinthecontextofthedatabaseitwascreatedin.However,whendroppingadatabase,anytasksthatwerecreatedinthecontextofthisdatabasewillremainactive.Itisthereforesensibletofirstunregisterallactivetasksforadatabasebeforedroppingthedatabase.
TasksregisteredinArangoDBwillbeexecuteduntiltheservergetsshutdownorrestarted.Afterarestartoftheserver,anyuser-definedone-shotorperiodictaskswillbelost.
CommandsforWorkingwithTasksArangoDBprovidesthefollowingcommandsforworkingwithtasks.Allcommandscanbeaccessedviathetasksmodule,whichcanbeloadedlikethis:
require("@arangodb/tasks")
PleasenotethatthetasksmoduleisavailableinsidetheArangoDBserveronly.ItcannotbeusedfromtheArangoShellorArangoDB'swebinterface.
Registeratask
Toregisteratask,theJavaScriptsnippetorfunctionneedstobespecifiedinadditiontotheexecutionfrequency.Optionally,ataskcanhaveanidandaname.Ifnoidisspecified,itwillbeauto-assignedforanewtask.Thetaskidisalsothemeanstoaccessorunregisteratasklater.Tasknamesareinformationalonly.Theycanbeusedtomakeataskdistinguishablefromothertasksalsorunningontheserver.
Thefollowingserver-sidecommandsregisteratask.ThecommandtobeexecutedisaJavaScriptstringsnippetwhichprintsamessagetotheserver'slogfile:
consttasks=require("@arangodb/tasks");
tasks.register({
id:"mytask-1",
name:"thisisasnippettask",
period:15,
command:"require('console').log('hellofromsnippettask');"
});
Theabovehasregisterataskwithidmytask-1,whichwillbeexecutedevery15secondsontheserver.Thetaskwillwritealogmessagewheneveritisinvoked.
TaskscanalsobesetupusingaJavaScriptcallbackfunctionlikethis:
TaskManagement
626
consttasks=require("@arangodb/tasks");
tasks.register({
id:"mytask-2",
name:"thisisafunctiontask",
period:15,
command:function(){
require('console').log('hellofromfunctiontask');
}
});
Itisimportanttonotethatthecallbackfunctionislateboundandwillbeexecutedinadifferentcontextthaninthecreationcontext.Thecallbackfunctionmustthereforenotaccessanyvariablesdefinedoutsideofitsownscope.Thecallbackfunctioncanstilldefineanduseitsownvariables.
Topassparameterstoatask,theparamsattributecanbesetwhenregisteringatask.NotethattheparametersarelimitedtodatatypesusableinJSON(meaningnocallbackfunctionscanbepassedasparametersintoatask):
consttasks=require("@arangodb/tasks");
tasks.register({
id:"mytask-3",
name:"thisisaparametertask",
period:15,
command:function(params){
vargreeting=params.greeting;
vardata=JSON.stringify(params.data);
require('console').log('%sfromparametertask:%s',greeting,data);
},
params:{greeting:"hi",data:"howareyou?"}
});
Registeringaone-shottaskworksthesameway,exceptthattheperiodattributemustbeomitted.Ifperiodisomitted,thenthetaskwillbeexecutedjustonce.Thetaskinvocationdelaycanoptionallybespecifiedwiththeoffsetattribute:
consttasks=require("@arangodb/tasks");
tasks.register({
id:"mytask-once",
name:"thisisaone-shottask",
offset:10,
command:function(params){
require('console').log('youwillseemejustonce!');
}
});
Note:Whenspecifyinganoffsetvalueof0,ArangoDBwillinternallyaddaverysmallvaluetotheoffsetsowillbeslightlygreaterthanzero.
Unregisteratask
Afterataskhasbeenregistered,itcanbeunregisteredusingitsid:
consttasks=require("@arangodb/tasks");
tasks.unregister("mytask-1");
Notethatunregisteringanon-existingtaskwillthrowanexception.
ListalltasksTogetanoverviewofwhichtasksareregistered,thereisthegetmethod.Ifthegetmethodiscalledwithoutanyarguments,itwillreturnanarrayofalltasks:
TaskManagement
627
consttasks=require("@arangodb/tasks");
tasks.get();
Ifgetiscalledwithataskidargument,itwillreturninformationaboutthisparticulartask:
consttasks=require("@arangodb/tasks");
tasks.get("mytask-3");
Thecreatedattributeofataskrevealswhenataskwascreated.ItisreturnedasaUnixtimestamp.
TaskManagement
628
DeprecatedFeatureslistedinthissectionshouldnolongerbeused,becausetheyareconsideredobsoleteandmaygetremovedinafuturerelease.Theyarecurrentlykeptforbackwardcompatibility.Thereareusuallybetteralternativestoreplacetheoldfeatureswith:
SimpleQueries:Ideomaticinterfaceinarangoshtoperformtrivialqueries.TheyaresupersededbyAQLqueries,whichcanalsoberuninarangosh.AQLisalanguageonitsownandwaymorepowerfulthanSimpleQueriescouldeverbe.Infact,the(stillsupported)SimpleQueriesaretranslatedinternallytoAQL,thentheAQLqueryisoptimizedandrunagainstthedatabaseinrecentversions,becauseofbetterperformanceandreducedmaintenancecomplexity.
Actions:SnippetsofJavaScriptcodeontheserver-sideforminimalcustomendpoints.SincetheFoxxrevampin3.0,itbecamereallyeasytowriteFoxxMicroservices,whichallowyoutodefinecustomendpointsevenwithcomplexbusinesslogic.
Deprecated
629
SimpleQueries
ItisrecommendedtouseAQLinstead.
Simplequeriescanbeusedifthequeryconditionisstraightforward,i.e.,adocumentreference,alldocuments,aquery-by-example,orasimplegeoquery.Inasimplequeryyoucanspecifyexactlyonecollectionandonequerycriteria.InthefollowingsectionswedescribetheJavaScriptshellinterfaceforsimplequeries,whichyoucanusewithintheArangoDBshellandwithinactionsandtransactions.ForotherlanguagesseethecorrespondinglanguageAPIdocumentation.
YoucanfindalistofqueriesatCollectionMethods.
SimpleQueries
630
SequentialAccessandCursorsIfaqueryreturnsacursor,thenyoucanusehasNextandnexttoiterateovertheresultsetortoArraytoconvertittoanarray.
Ifthenumberofqueryresultsisexpectedtobebig,itispossibletolimittheamountofdocumentstransferredbetweentheserverandtheclienttoaspecificvalue.ThisvalueiscalledbatchSize.ThebatchSizecanoptionallybesetbeforeorwhenasimplequeryisexecuted.Iftheserverhasmoredocumentsthanshouldbereturnedinasinglebatch,theserverwillsetthehasMoreattributeintheresult.Itwillalsoreturntheidoftheserver-sidecursorintheidattributeintheresult.ThisidcanbeusedwiththecursorAPItofetchanyoutstandingresultsfromtheserveranddisposetheserver-sidecursorafterwards.
TheinitialbatchSizevaluecanbesetusingthesetBatchSizemethodthatisavailableforeachtypeofsimplequery,orwhenthesimplequeryisexecutedusingitsexecutemethod.IfnobatchSizevalueisspecified,theserverwillpickareasonabledefaultvalue.
HasNext
checksifthecursorisexhaustedcursor.hasNext()
ThehasNextoperatorreturnstrue,thenthecursorstillhasdocuments.Inthiscasethenextdocumentcanbeaccessedusingthenextoperator,whichwilladvancethecursor.
Examples
arangosh>vara=db.five.all();
arangosh>while(a.hasNext())print(a.next());
showexecutionresults
Next
returnsthenextresultdocumentcursor.next()
IfthehasNextoperatorreturnstrue,thentheunderlyingcursorofthesimplequerystillhasdocuments.Inthiscasethenextdocumentcanbeaccessedusingthenextoperator,whichwilladvancetheunderlyingcursor.Ifyouusenextonanexhaustedcursor,thenundefinedisreturned.
Examples
arangosh>db.five.all().next();
showexecutionresults
SetBatchsize
setsthebatchsizeforanyfollowingrequestscursor.setBatchSize(number)
Setsthebatchsizeforqueries.Thebatchsizedetermineshowmanyresultsareatmosttransferredfromtheservertotheclientinonechunk.
GetBatchsize
returnsthebatchsizecursor.getBatchSize()
Returnsthebatchsizeforqueries.Ifthereturnedvalueisundefined,theserverwilldetermineasensiblebatchsizeforanyfollowingrequests.
ExecuteQuery
executesaqueryquery.execute(batchSize)
SimpleQueries
631
Executesasimplequery.IftheoptionalbatchSizevalueisspecified,theserverwillreturnatmostbatchSizevaluesinoneroundtrip.ThebatchSizecannotbeadjustedafterthequeryisfirstexecuted.
Note:Thereisnoneedtoexplicitlycalltheexecutemethodifanothermeansoffetchingthequeryresultsischosen.Thefollowingtwoapproachesleadtothesameresult:
arangosh>result=db.users.all().toArray();
arangosh>q=db.users.all();q.execute();result=[];while(q.hasNext()){
result.push(q.next());}
showexecutionresultsThefollowingtwoalternativesbothuseabatchSizeandreturnthesameresult:
arangosh>q=db.users.all();q.setBatchSize(20);q.execute();while(q.hasNext()){
print(q.next());}
arangosh>q=db.users.all();q.execute(20);while(q.hasNext()){print(q.next());}
showexecutionresults
Dispose
disposestheresultcursor.dispose()
Ifyouarenolongerinterestedinanyfurtherresults,youshouldcalldisposeinordertofreeanyresourcesassociatedwiththecursor.Aftercallingdisposeyoucannolongeraccessthecursor.
Count
countsthenumberofdocumentscursor.count()
Thecountoperatorcountsthenumberofdocumentintheresultsetandreturnsthatnumber.Thecountoperatorignoresanylimitsandreturnsthetotalnumberofdocumentsfound.
Note:Notallsimplequeriessupportcounting.Inthiscasenullisreturned.
cursor.count(true)
Iftheresultsetwaslimitedbythelimitoperatorordocumentswereskipedusingtheskipoperator,thecountoperatorwithargumenttruewillusethenumberofelementsinthefinalresultset-afterapplyinglimitandskip.
Note:Notallsimplequeriessupportcounting.Inthiscasenullisreturned.
Examples
Ignoreanylimit:
arangosh>db.five.all().limit(2).count();
null
Countinganylimitorskip:
arangosh>db.five.all().limit(2).count(true);
2
SimpleQueries
632
Pagination
ItisrecommendedtouseAQLinstead,seetheLIMIToperation.
If,forexample,youdisplaytheresultofausersearch,thenyouareingeneralnotinterestedinthecompletedresultset,butonlythefirst10orsodocuments.Ormaybethenext10documentsforthesecondpage.Inthiscase,youcantheskipandlimitoperators.TheseoperatorsworklikeLIMITinMySQL.
skipusedtogetherwithlimitcanbeusedtoimplementpagination.Theskipoperatorskipsoverthefirstndocuments.So,inordertocreateresultpageswith10resultdocumentsperpage,youcanuseskip(n*10).limit(10)toaccessthe10documentsonthenthpage.Thisresultshouldbesorted,sothatthepaginationworksinapredicableway.
Limit
limitquery.limit(number)
Limitsaresulttothefirstnumberdocuments.Specifyingalimitof0willreturnnodocumentsatall.Ifyoudonotneedalimit,justdonotaddthelimitoperator.Thelimitmustbenon-negative.
Ingeneraltheinputtolimitshouldbesorted.Otherwiseitwillbeunclearwhichdocumentswillbeincludedintheresultset.
Examples
arangosh>db.five.all().toArray();
arangosh>db.five.all().limit(2).toArray();
showexecutionresults
Skip
skipquery.skip(number)
Skipsthefirstnumberdocuments.Ifnumberispositive,thenthisnumberofdocumentsareskippedbeforereturningthequeryresults.
Ingeneraltheinputtoskipshouldbesorted.Otherwiseitwillbeunclearwhichdocumentswillbeincludedintheresultset.
Note:usingnegativeskipvaluesisdeprecatedasofArangoDB2.6andwillnotbesupportedinfutureversionsofArangoDB.
Examples
arangosh>db.five.all().toArray();
arangosh>db.five.all().skip(3).toArray();
showexecutionresults
SimpleQueries
633
ModificationQueries
ItisrecommendedtouseAQLinstead,seeDataModificationQueries.
ArangoDBalsoallowsremoving,replacing,andupdatingdocumentsbasedonanexampledocument.Everydocumentinthecollectionwillbecomparedagainstthespecifiedexampledocumentandbedeleted/replaced/updatedifallattributesmatch.
Thesemethodshouldbeusedwithcautionastheyareintendedtoremoveormodifylotsofdocumentsinacollection.
Allmethodscanoptionallyberestrictedtoaspecificnumberofoperations.However,ifalimitisspecificbutislessthanthenumberofmatches,itwillbeundefinedwhichofthematchingdocumentswillgetremoved/modified.RemovebyExample,ReplacebyExampleandUpdatebyExamplearedescribedwithexamplesinthesubchapterCollectionMethods.
SimpleQueries
634
GeoQueries
ItisrecommendedtouseAQLinstead,seeGeofunctions.
TheArangoDBallowstoselectdocumentsbasedongeographiccoordinates.Inorderforthistowork,ageo-spatialindexmustbedefined.ThisindexwilluseaveryelaboratealgorithmtolookupneighborsthatisamagnitudefasterthanasimpleR*index.
Ingeneralageocoordinateisapairoflatitudeandlongitude,whichmustbothbespecifiedasnumbers.Ageoindexcanbecreatedoncoordinatesthatarestoredinasinglelistattributewithtwoelementslike[-10,+30](latitudefirst,followedbylongitude)oroncoordinatesstoredintwoseparateattributes.
Forexample,toindexthefollowingdocuments,anindexcanbecreatedonthepositionattributeofthedocuments:
db.test.save({position:[-10,30]});
db.test.save({position:[10,45.5]});
db.test.ensureIndex({type:"geo",fields:["position"]});
Ifcoordinatesarestoredintwodistinctattributes,theindexmustbecreatedonthetwoattributes:
db.test.save({latitude:-10,longitude:30});
db.test.save({latitude:10,longitude:45.5});
db.test.ensureIndex({type:"geo",fields:["latitude","longitude"]});
Inordertofindalldocumentswithinagivenradiusaroundacoordinateusethewithinoperator.Inordertofindalldocumentsnearagivendocumentusethenearoperator.
Itispossibletodefinemorethanonegeo-spatialindexpercollection.Inthiscaseyoumustgiveahintusingthegeooperatorwhichofindexesshouldbeusedinaquery.
Near
constructsanearqueryforacollectioncollection.near(latitude,longitude)Thereturnedlistissortedaccordingtothedistance,withthenearestdocumenttothecoordinate(latitude,longitude)comingfirst.Ifthereareneardocumentsofequaldistance,documentsarechosenrandomlyfromthissetuntilthelimitisreached.Itispossibletochangethelimitusingthelimitoperator.Inordertousethenearoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesforthedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.Note:neardoesnotsupportnegativeskips.//However,youcanstilluselimitfollowedtoskip.collection.near(latitude,longitude).limit(limit)Limitstheresulttolimitdocumentsinsteadofthedefault100.Note:Unlikewithmultipleexplicitlimits,limitwillraisetheimplicitdefaultlimitimposedbywithin.collection.near(latitude,longitude).distance()Thiswilladdanattributedistancetoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.collection.near(latitude,longitude).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thenearsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionusingthenearoperatoristousetheAQLNEARfunctioninanAQLqueryasfollows:
FORdocINNEAR(@@collection,@latitude,@longitude,@limit)
RETURNdoc
Examples
Togetthenearesttwolocations:
arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});
arangosh>for(vari=-90;i<=90;i+=10){
SimpleQueries
635
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({
........>name:"Name/"+i+"/"+j,
........>loc:[i,j]});
........>}}
arangosh>db.geo.near(0,0).limit(2).toArray();
showexecutionresultsIfyouneedthedistanceaswell,thenyoucanusethedistanceoperator:
arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});
arangosh>for(vari=-90;i<=90;i+=10){
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({
........>name:"Name/"+i+"/"+j,
........>loc:[i,j]});
........>}}
arangosh>db.geo.near(0,0).distance().limit(2).toArray();
showexecutionresults
Within
constructsawithinqueryforacollectioncollection.within(latitude,longitude,radius)Thiswillfindalldocumentswithinagivenradiusaroundthecoordinate(latitude,longitude).Thereturnedarrayissortedbydistance,beginningwiththenearestdocument.Inordertousethewithinoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesforthedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.collection.within(latitude,longitude,radius).distance()Thiswilladdanattribute_distancetoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.collection.within(latitude,longitude,radius).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thewithinsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionusingthewithinoperatoristousetheAQLWITHINfunctioninanAQLqueryasfollows:
FORdocINWITHIN(@@collection,@latitude,@longitude,@radius,@distanceAttributeName)
RETURNdoc
Examples
Tofindalldocumentswithinaradiusof2000kmuse:
arangosh>for(vari=-90;i<=90;i+=10){
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});}}
arangosh>db.geo.within(0,0,2000*1000).distance().toArray();
showexecutionresults
Geo
constructsageoindexselectioncollection.geo(location-attribute)Looksupageoindexdefinedonattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Thisisusefulforcollectionswithmultipledefinedgeoindexes.collection.geo(location_attribute,true)Looksupageoindexonacompoundattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.collection.geo(latitude_attribute,longitude_attribute)Looksupageoindexdefinedonthetwoattributeslatitude_attributeandlongitude-attribute.Returnsageoindexobjectifanindexwas
SimpleQueries
636
found.Thenearorwithinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thegeosimplequeryhelperfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforrunninggeoqueriesistousetheirAQLequivalents.
Examples
Assumeyouhavealocationstoredaslistintheattributehomeandadestinationstoredintheattributework.Thenyoucanusethegeooperatortoselectwhichgeo-spatialattributes(andthuswhichindex)touseinanearquery.
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.complex.save({name:"Name/"+i+"/"+j,
........>home:[i,j],
........>work:[-i,-j]});
........>}
........>}
........>
arangosh>db.complex.near(0,170).limit(5);
arangosh>db.complex.ensureIndex({type:"geo",fields:["home"]});
arangosh>db.complex.near(0,170).limit(5).toArray();
arangosh>db.complex.geo("work").near(0,170).limit(5);
arangosh>db.complex.ensureIndex({type:"geo",fields:["work"]});
arangosh>db.complex.geo("work").near(0,170).limit(5).toArray();
showexecutionresults
Relatedtopics
OtherArangoDBgeographicfeaturesaredescribedin:
AQLGeofunctionsGeoindexes
SimpleQueries
637
Fulltextqueries
ItisrecommendedtouseAQLinstead,seeFulltextfunctions.
ArangoDBallowstorunqueriesontextcontainedindocumentattributes.Tousethis,afulltextindexmustbedefinedfortheattributeofthecollectionthatcontainsthetext.Creatingtheindexwillparsethetextinthespecifiedattributeforalldocumentsofthecollection.Onlydocumentswillbeindexedthatcontainatextualvalueintheindexedattribute.Forsuchdocuments,thetextvaluewillbeparsed,andtheindividualwordswillbeinsertedintothefulltextindex.
Whenafulltextindexexists,itcanbequeriedusingafulltextquery.
Fulltext
queriesthefulltextindexcollection.fulltext(attribute,query)
Thefulltextsimplequeryfunctionsperformsafulltextsearchonthespecifiedattributeandthespecifiedquery.
Detailsaboutthefulltextquerysyntaxcanbefoundbelow.
Note:thefulltextsimplequeryfunctionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforexecutingfulltextqueriesistouseanAQLqueryusingtheFULLTEXTAQLfunctionasfollows:
FORdocINFULLTEXT(@@collection,@attributeName,@queryString,@limit)
RETURNdoc
Examples
arangosh>db.emails.ensureFulltextIndex("content");
arangosh>db.emails.save({content:
........>"HelloAlice,howareyoudoing?Regards,Bob"});
arangosh>db.emails.save({content:
........>"HelloCharlie,doAliceandBobknowaboutit?"});
arangosh>db.emails.save({content:"Ithinktheydon'tknow.Regards,Eve"});
arangosh>db.emails.fulltext("content","charlie,|eve").toArray();
showexecutionresults
FulltextSyntax:
Inthesimplestform,afulltextquerycontainsjustthesoughtword.Ifmultiplesearchwordsaregiveninaquery,theyshouldbeseparatedbycommas.AllsearchwordswillbecombinedwithalogicalANDbydefault,andonlysuchdocumentswillbereturnedthatcontainallsearchwords.Thisdefaultbehaviorcanbechangedbyprovidingtheextracontrolcharactersinthefulltextquery,whichare:
+:logicalAND(intersection)|:logicalOR(union)-:negation(exclusion)
Examples:
"banana":searchesfordocumentscontaining"banana""banana,apple":searchesfordocumentscontainingboth"banana"AND"apple""banana,|orange":searchesfordocumentscontainingeither"banana"OR"orange"ORboth"banana,-apple":searchesfordocumentsthatcontains"banana"butNOT"apple".
Logicaloperatorsareevaluatedfromlefttoright.
SimpleQueries
638
Eachsearchwordcanoptionallybeprefixedwithcomplete:orprefix:,withcomplete:beingthedefault.Thisallowssearchingforcompletewordsorforwordprefixes.Suffixsearchesoranyotherformsarepartial-wordmatchingarecurrentlynotsupported.
Examples:
"complete:banana":searchesfordocumentscontainingtheexactword"banana""prefix:head":searchesfordocumentswithwordsthatstartwithprefix"head""prefix:head,banana":searchesfordocumentscontainwordsstartingwithprefix"head"andthatalsocontaintheexactword"banana".
Completematchandprefixsearchoptionscanbecombinedwiththelogicaloperators.
Pleasenotethatonlywordswithaminimumlengthwillgetindexed.Thisminimumlengthcanbedefinedwhencreatingthefulltextindex.Forwordstokenization,thelibicutextboundaryanalysisisused,whichtakesintoaccountthedefaultasdefinedatserverstartup(--server.default-languagestartupoption).Generally,thewordboundaryanalysiswillfilteroutpunctuationbutwillnotdomuchmore.
Especiallynowordnormalization,stemming,orsimilarityanalysiswillbeperformedwhenindexingorsearching.Ifanyofthesefeaturesisrequired,itissuggestedthattheuserdoesthetextnormalizationontheclientside,andprovidesforeachdocumentanextraattributecontainingjustacomma-separatedlistofnormalizedwords.Thisattributecanthenbeindexedwithafulltextindex,andtheusercansendfulltextqueriesforthisindex,withthefulltextqueriesalsocontainingthestemmedornormalizedversionsofwordsasrequiredbytheuser.
SimpleQueries
639
ArangoDB'sActions
ItisrecommendedtouseFoxxinstead.
IntroductiontoUserActions
InsomewaysthecommunicationlayeroftheArangoDBserverbehaveslikeaWebserver.UnlikeaWebserver,itnormallyrespondstoHTTPrequestsbydeliveringJSONobjects.Remember,documentsinthedatabasearejustJSONobjects.So,mostofthetimetheHTTPresponsewillcontainaJSONdocumentfromthedatabaseasbody.YoucanextractthedocumentsstoredinthedatabaseusingHTTPGET.YoucanstoredocumentsusingHTTPPOST.
However,thereissomethingmore.Youcanwritesmallsnippets-socalledactions-toextendthedatabase.TheideaofactionsisthatsometimesitisbettertostorepartsofthebusinesslogicwithinArangoDB.
Thesimplestexampleistheageofaperson.Assumeyoustoreinformationaboutpeopleinyourdatabase.Itisananti-patterntostoretheage,becauseitchangeseverynowandthen.Therefore,younormallystorethebirthdayandlettheclientdecidewhattodowithit.However,ifyouhavemanydifferentclients,itmightbeeasiertoenrichthepersondocumentwiththeageusingactionsonceontheserverside.
Or,forinstance,ifyouwanttoapplysomestatisticstolargedata-setsandyoucannoteasilyexpressthisasquery.Youcandefineaactioninsteadoftransferringthewholedatatotheclientanddothecomputationontheclient.
Actionsarealsousefulifyouwanttorestrictandfilterdataaccordingtosomecomplexpermissionsystem.
TheArangoDBservercandeliverallkindsofinformation,JSONbeingonlyonepossibleformat.YoucanalsogenerateHTMLorimages.However,aWebserverisnormallybettersuitedforthetaskasitalsoimplementsvariouscachingstrategies,languageselection,compressionandsoon.Havingsaidthat,therearestillsituationswhereitmightbesuitabletousetheArangoDBtodeliverHTMLpages-staticordynamic.Asimpleexampleisthebuilt-inadministrationinterface.YoucanaccessitusinganymodernbrowserandthereisnoneedforaseparateApacheorIIS.
IngeneralyouwilluseFoxxtoeasilyextendthedatabasewithbusinesslogic.Foxxprovidesansimpletouseinterfacetoactions.
Thefollowingsectionswillexplainthelow-levelactionswithinArangoDBonwhichFoxxisbuiltandshowhowtodefinethem.TheexamplesstartwithdeliveringstaticHTMLpages-evenifthisisnottheprimaryuse-caseforactions.ThelatersectionswillthenshowyouhowtocodesomepiecesofyourbusinesslogicandreturnJSONobjects.
TheinterfaceislooselymodeledaftertheJavaScriptclassesforHTTPrequestandresponsesfoundinnode.jsandthemiddleware/routingaspectsofconnect.jsandexpress.js.
Notethatunlikenode.js,ArangoDBismulti-threadedandthereisnoeasywaytosharestatebetweenqueriesinsidetheJavaScriptengine.Ifsuchstateinformationisrequired,youneedtousethedatabaseitself.
Actions
640
AHelloWorldExample
TheclientAPIorbrowsersendsaHTTPrequesttotheArangoDBserverandtheserverreturnsaHTTPresponsetotheclient.AHTTPrequestconsistsofamethod,normallyGETorPOSTwhenusingabrowser,andarequestpathlike/hello/world.ForarealWebserverthereareazillionofotherthingtoconsider,wewillignorethisforthemoment.TheHTTPresponsecontainsacontenttype,describinghowtointerpretthereturneddata,andthedataitself.
Inthefollowingexample,wewanttodefineanactioninArangoDB,sothattheserverreturnstheHTMLdocument
<html>
<body>
HelloWorld
</body>
</html>
ifaskedGET/hello/world.
Theserverneedstoknowwhatfunctiontocallorwhatdocumenttodeliverifitreceivesarequest.Thisiscalledrouting.AlltheroutinginformationofArangoDBisstoredinacollection_routing.Eachentryinthiscollectionsdescribeshowtodealwithaparticularrequestpath.
Fortheaboveexample,addthefollowingdocumenttothe_routingcollection:
arangosh>db._routing.save({
........>url:{
........>match:"/hello/world"
........>},
........>content:{
........>contentType:"text/html",
........>body:"<html><body>HelloWorld</body></html>"
........>}
........>});
showexecutionresultsInordertoactivatethenewrouting,youmusteitherrestarttheserverorcalltheinternalreloadfunction.
arangosh>require("internal").reloadRouting()
Nowusethebrowserandaccesshttp://localhost:8529/hello/world
YoushouldseetheHelloWorldinourbrowser:
shell>curl--dump-http://localhost:8529/hello/world
HTTP/1.1200OK
content-type:text/html
x-content-type-options:nosniff
"HelloWorld"
MatchingaURLTherearealotofoptionsfortheurlattribute.Ifyoudefinedifferentroutingforthesamepath,thenthefollowingsimpleruleisappliedinordertodeterminewhichmatchwins:Iftherearetwomatches,thenthemorespecificwins.I.e,ifthereisawildcardmatchandanexactmatch,theexactmatchispreferred.Ifthereisashortandalongmatch,thelongermatchwins.
Actions
641
ExactMatch
Ifthedefinitionis
{
url:{
match:"/hello/world"
}
}
thenthematchmustbeexact.Onlytherequestfor/hello/worldwillmatch,everythingelse,e.g./hello/world/myor/hello/world2,willnotmatch.
Thefollowingdefinitionisashort-cutforanexactmatch.
{
url:"/hello/world"
}
Note:WhilethetwodefinitionswillresultinthesameURLmatching,thereisasubtledifferencebetweenthem:
Theformerdefinition(definingurlasanobjectwithamatchattribute)willresultintheURLbeingaccessibleviaallsupportedHTTPmethods(e.g.GET,POST,PUT,DELETE,...),whereasthelatterdefinition(providingastringurlattribute)willresultintheURLbeingaccessibleviaHTTPGETandHTTPHEADonly,withallotherHTTPmethodsbeingdisabled.CallingaURLwithanunsupportedordisabledHTTPmethodwillresultinanHTTP501(notimplemented)error.
PrefixMatch
Ifthedefinitionis
{
url:{
match:"/hello/world/*"
}
}
thenthematchcanbeaprefixmatch.Therequestsfor/hello/world,/hello/world/my,and/hello/world/how/are/youwillallmatch.However/hello/world2doesnotmatch.PrefixmatcheswithinaURLpart,i.e./hello/world*,arenotallowed.Thewildcardmustoccurattheend,i.e.
/hello/*/world
isalsodisallowed.
Ifyoudefinetworoutes
{url:{match:"/hello/world/*"}}
{url:{match:"/hello/world/emil"}}
thenthesecondroutewillbeusedfor/hello/world/emilbecauseitismorespecific.
ParameterizedMatch
Aparameterizedmatchissimilartoaprefixmatch,buttheparametersarealsoallowedinsidetheURLpath.
Ifthedefinitionis
{
url:{
match:"/hello/:name/world"
}
}
Actions
642
thentheURLmusthavethreeparts,thefirstpartbeinghelloandthethirdpartworld.Forexample,/hello/emil/worldwillmatch,while/hello/emil/meyer/worldwillnot.
ConstraintMatch
Aconstraintmatchissimilartoaparameterizedmatch,buttheparameterscancarryconstraints.
Ifthedefinitionis
{
url:{
match:"/hello/:name/world",
constraint:{
name:"/[a-z]+/"
}
}
}
thentheURLmusthavethreeparts,thefirstpartbeinghelloandthethirdpartworld.Thesecondpartmustbealllowercase.
ItispossibletousemorethenoneconstraintforthesameURLpart.
{
url:{
match:"/hello/:name|:id/world",
constraint:{
name:"/[a-z]+/",id:"/[0-9]+/"
}
}
}
OptionalMatch
Anoptionalmatchissimilartoaparameterizedmatch,butthelastparameterisoptional.
Ifthedefinitionis
{
url:{
match:"/hello/:name?",
constraint:{
name:"/[a-z]+/"
}
}
}
thentheURL/helloand/hello/emilwillmatch.
Ifthedefinitionsare
{url:{match:"/hello/world"}}
{url:{match:"/hello/:name",constraint:{name:"/[a-z]+/"}}}
{url:{match:"/hello/*"}}
thentheURL/hello/worldwillbematchedbythefirstroute,becauseitisthemostspecific.TheURL/hello/youwillbematchedbythesecondroute,becauseitismorespecificthantheprefixmatch.
MethodRestriction
YoucanrestrictthematchtospecificHTTPmethods.
Ifthedefinitionis
{
url:{
Actions
643
match:"/hello/world",
methods:["post","put"]
}
}
thenonlyHTTPPOSTandPUTrequestswillmatch.CallingwithadifferentHTTPmethodwillresultinanHTTP501error.
Pleasenotethatifurlisdefinedasasimplestring,thenonlytheHTTPmethodsGETandHEADwillbeallowed,anallothermethodswillbedisabled:
{
url:"/hello/world"
}
MoreonMatching
Rememberthatthemorespecificmatchwins.
Amatchwithoutparameterorwildcardismorespecificthanamatchwithparametersorwildcard.Amatchwithparameterismorespecificthanamatchwithawildcard.Ifthereismorethanoneparameter,specificityisappliedfromlefttoright.
Considerthefollowingdefinitions
arangosh>db._routing.save({
........>url:{match:"/hello/world"},
........>content:{contentType:"text/plain",body:"MatchNo1"}});
arangosh>db._routing.save({
........>url:{match:"/hello/:name",constraint:{name:"/[a-z]+/"}},
........>content:{contentType:"text/plain",body:"MatchNo2"}});
arangosh>db._routing.save({
........>url:{match:"/:something/world"},
........>content:{contentType:"text/plain",body:"MatchNo3"}});
arangosh>db._routing.save({
........>url:{match:"/hi/*"},
........>content:{contentType:"text/plain",body:"MatchNo4"}});
arangosh>require("internal").reloadRouting()
showexecutionresultsThen
shell>curl--dump-http://localhost:8529/hello/world
HTTP/1.1200OK
content-type:text/plain
x-content-type-options:nosniff
"MatchNo1"
shell>curl--dump-http://localhost:8529/hello/emil
HTTP/1.1200OK
content-type:text/plain
x-content-type-options:nosniff
"MatchNo2"
shell>curl--dump-http://localhost:8529/your/world
HTTP/1.1200OK
Actions
644
content-type:text/plain
x-content-type-options:nosniff
"MatchNo3"
shell>curl--dump-http://localhost:8529/hi/you
HTTP/1.1200OK
content-type:text/plain
x-content-type-options:nosniff
"MatchNo4"
Youcanwritethefollowingdocumentintothe_routingcollectiontotesttheaboveexamples.
{
routes:[
{url:{match:"/hello/world"},content:"route1"},
{url:{match:"/hello/:name|:id",constraint:{name:"/[a-z]+/",id:"/[0-9]+/"}},content:"route2"},
{url:{match:"/:something/world"},content:"route3"},
{url:{match:"/hello/*"},content:"route4"},
]
}
Actions
645
AHelloWorldExampleforJSON
Ifyouchangetheexampleslightly,thenaJSONobjectwillbedelivered.
arangosh>db._routing.save({
........>url:"/hello/json",
........>content:{
........>contentType:"application/json",
........>body:'{"hello":"world"}'
........>}
........>});
arangosh>require("internal").reloadRouting()
showexecutionresultsAgaincheckwithyourbrowserorcURLhttp://localhost:8529/hello/json
Dependingonyourbrowserandinstalledadd-onsyouwilleitherseetheJSONobjectoradownloaddialog.IfyourbrowserwantstoopenanexternalapplicationtodisplaytheJSONobject,youcanchangethecontentTypeto"text/plain" fortheexample.Thismakesiteasiertochecktheexampleusingabrowser.Orusecurltoaccesstheserver.
shell>curl--dump-http://localhost:8529/hello/json
HTTP/1.1200OK
content-type:application/json;charset=utf-8
x-content-type-options:nosniff
{
"hello":"world"
}
DeliveringContent
Therearealotofdifferentwaysonhowtodelivercontent.Wehavealreadyseenthesimplestone,wherestaticcontentisdelivered.Thefun,however,startswhendeliveringdynamiccontent.
StaticContent
Youcanspecifyabodyandacontent-type.
arangosh>db._routing.save({
........>url:"/hello/contentType",
........>content:{
........>contentType:"text/html",
........>body:"<html><body>HelloWorld</body></html>"
........>}
........>});
arangosh>require("internal").reloadRouting()
showexecutionresults
shell>curl--dump-http://localhost:8529/hello/contentType
Actions
646
HTTP/1.1200OK
content-type:text/html
x-content-type-options:nosniff
"HelloWorld"
Ifthecontenttypeistext/plainthenyoucanusetheshort-cut
{
content:"HelloWorld"
}
ASimpleAction
Thesimplestdynamicactionis:
{
action:{
do:"@arangodb/actions/echoRequest"
}
}
Itisnotadvisabletostorefunctionsdirectlyintheroutingtable.Itisbettertocallfunctionsdefinedinmodules.IntheaboveexamplethefunctioncanbeaccessedfromJavaScriptas:
require("@arangodb/actions").echoRequest
ThefunctionechoRequestispre-defined.Ittakestherequestobjectsandechositintheresponse.
Thesignatureofsuchafunctionmustbe
function(req,res,options,next)
Examples
arangosh>db._routing.save({
........>url:"/hello/echo",
........>action:{
........>do:"@arangodb/actions/echoRequest"
........>}
........>});
showexecutionresultsReloadtheroutingandcheckhttp://127.0.0.1:8529/hello/echo
Youshouldseesomethinglike
arangosh>arango.GET("/hello/echo")
showexecutionresults
Therequestmightcontainpath,prefix,suffix,andurlParametersattributes.pathisthecompletepathassuppliedbytheuserandalwaysavailable.Ifaprefixwasmatched,thenthisprefixisstoredintheattributeprefixandtheremainingURLpartsarestoredasanarrayinsuffix.Ifoneormoreparameterswerematched,thentheparametervaluesarestoredinurlParameters.
Actions
647
Forexample,iftheurldescriptionis
{
url:{
match:"/hello/:name/:action"
}
}
andyourequestthepath/hello/emil/jump,thentherequestobjectwillcontainthefollowingattribute
urlParameters:{
name:"emil",
action:"jump"
}
ActionController
Asanalternativetothesimpleaction,youcanusecontrollers.Acontrollerisamodule,definesthefunctionget,put,post,delete,head,patch.Ifarequestofthecorrespondingtypeismatched,thefunctionwillbecalled.
Examples
arangosh>db._routing.save({
........>url:"/hello/echo",
........>action:{
........>controller:"@arangodb/actions/echoController"
........>}
........>});
showexecutionresultsReloadtheroutingandcheckhttp://127.0.0.1:8529/hello/echo:
arangosh>arango.GET("/hello/echo")
showexecutionresults
PrefixActionController
Thecontrollerisselectedwhenthedefinitionisread.Thereisamoreflexible,butslowerandmaybeinsecurevariant,theprefixcontroller.
Assumethattheurlisaprefixmatch
{
url:{
match:/hello/*"
}
}
Youcanuse
{
action:{
prefixController:"@arangodb/actions"
}
}
todefineaprefixcontroller.IftheURL/hello/echoControllerisgiven,thenthemodule@arangodb/actions/echoControllerisused.
Ifyouuseaprefixcontroller,youshouldmakecertainthatnounwantedactionsareavailableundertheprefix.
Actions
648
Thedefinition
{
action:"@arangodb/actions"
}
isashort-cutforaprefixcontrollerdefinition.
FunctionAction
Youcanalsostoreafunctiondirectlyintheroutingtable.
Examples
arangosh>db._routing.save({
........>url:"/hello/echo",
........>action:{
........>callback:"function(req,res){res.statusCode=200;res.body='Hello'}"
........>}
........>});
showexecutionresults
arangosh>arango.GET("hello/echo")
arangosh>db._query("FORrouteIN_routingFILTERroute.url=='/hello/echo'REMOVEroute
in_routing")
[objectArangoQueryCursor,count:0,cached:false,hasMore:false]
arangosh>require("internal").reloadRouting()
RequestsandResponses
Thecontrollermustdefinehandlerfunctionswhichtakearequestobjectandfilltheresponseobject.
AverysimpleexampleisthefunctionechoRequestdefinedinthemodule@arangodb/actions.
function(req,res,options,next){
varresult;
result={request:req,options:options};
res.responseCode=exports.HTTP_OK;
res.contentType="application/json";
res.body=JSON.stringify(result);
}
Installitvia:
arangosh>db._routing.save({
........>url:"/echo",
........>action:{
........>do:"@arangodb/actions/echoRequest"
........>}
........>})
showexecutionresultsReloadtheroutingandcheckhttp://127.0.0.1:8529/hello/echo
Youshouldseesomethinglike
Actions
649
arangosh>arango.GET("/hello/echo")
arangosh>db._query("FORrouteIN_routingFILTERroute.url=='/hello/echo'REMOVEroute
in_routing")
arangosh>require("internal").reloadRouting()
showexecutionresultsYoumayalsopassoptionstothecalledfunction:
arangosh>db._routing.save({
........>url:"/echo",
........>action:{
........>do:"@arangodb/actions/echoRequest",
........>options:{
........>"Hello":"World"
........>}
........>}
........>});
showexecutionresultsYounowseetheoptionsintheresult:
arangosh>arango.GET("/echo")
arangosh>db._query("FORrouteIN_routingFILTERroute.url=='/echo'REMOVEroutein
_routing")
arangosh>require("internal").reloadRouting()
showexecutionresults
Actions
650
ModifyingRequestandResponse
Aswe'veseeninthepreviousexamples,actionsgetcalledwiththerequestandresponseobjects(namedreqandresintheexamples)passedasparameterstotheirhandlerfunctions.
ThereqobjectcontainstheincomingHTTPrequest,whichmightormightnothavebeenmodifiedbyapreviousaction(ifactionswerechained).
Ahandlercanmodifytherequestobjectinplaceifdesired.Thismightbeusefulwhenwritingmiddleware(seebelow)thatisusedtointerceptincomingrequests,modifythemandpassthemtotheactualhandlers.
Whilemodifyingtherequestobjectmightnotbethatrelevantfornon-middlewareactions,modifyingtheresponseobjectdefinitelyis.Modifyingtheresponseobjectisanaction'sonlywaytoreturndatatothecalleroftheaction.
We'vealreadyseenhowtosettheHTTPstatuscode,thecontenttype,andtheresultbody.Theresobjecthasthefollowingpropertiesforthese:
contentType:MIMEtypeofthebodyasdefinedintheHTTPstandard(e.g.text/html,text/plain,application/json,...)responsecode:theHTTPstatuscodeoftheresponseasdefinedintheHTTPstandard.Commonvaluesforactionsthatsucceedare200or201.PleaserefertotheHTTPstandardformoreinformation.body:theactualresponsedata
Tosetormodifyarbitraryheadersoftheresponseobject,theheaderspropertycanbeused.Forexample,toaddauser-definedheadertotheresponse,thefollowingcodewilldo:
res.headers=res.headers||{};//headersmightormightnotbepresent
res.headers['X-Test']='someValue';//setheaderX-Testto"someValue"
ThiswillsettheadditionalHTTPheaderX-TesttovaluesomeValue.Otherheaderscanbesetaswell.NotethatArangoDBmightchangethecaseoftheheadernamestolowercasewhenassemblingtheoverallresponsethatissenttothecaller.
ItisnotnecessarytoexplicitlysetaContent-LengthheaderfortheresponseasArangoDBwillcalculatethecontentlengthautomaticallyandaddthisheaderitself.ArangoDBmightalsoaddaConnectionheaderitselftohandleHTTPkeep-alive.
ArangoDBalsosupportsautomatictransformationofthebodydatatoanotherformat.Currently,theonlysupportedtransformationsarebase64-encodingandbase64-decoding.Usingthetransformations,anactioncancreateabase64encodedbodyandstillletArangoDBsendthenon-encodedversion,forexample:
res.body='VGhpcyBpcyBhIHRlc3Q=';
res.transformations=res.transformations||[];//initialize
res.transformations.push('base64decode');//willbase64decodetheresponsebody
WhenArangoDBprocessestheresponse,itwillbase64-decodewhat'sinres.bodyandsettheHTTPheaderContent-Encoding:binary.Theoppositecanbeachievedwiththebase64encodetransformation:ArangoDBwillthenautomaticallybase64-encodethebodyandsetaContent-Encoding:base64HTTPheader.
WritingdynamicactionhandlersTowriteyourowndynamicactionhandlers,youmustputthemintomodules.
Modulesareameansoforganizingactionhandlersandmakingthemloadableunderspecificnames.
Tostart,we'lldefineasimpleactionhandlerinamodule/ownTest:
arangosh>db._modules.save({
........>path:"/db:/ownTest",
........>content:
........>"exports.do=function(req,res,options,next){"+
........>"res.body='test';"+
Actions
651
........>"res.responseCode=200;"+
........>"res.contentType='text/plain';"+
........>"};"
........>});
showexecutionresultsThisdoesnothingbutregisteradoactionhandlerinamodule/ownTest.Theactionhandlerisnotyetcallable,butmustbemappedtoaroutefirst.Tomaptheactiontotheroute/ourtest,executethefollowingcommand:
arangosh>db._routing.save({
........>url:"/ourtest",
........>action:{
........>controller:"db://ownTest"
........>}
........>});
arangosh>require("internal").reloadRouting()
showexecutionresultsNowusethebrowserorcURLandaccesshttp://localhost:8529/ourtest:
shell>curl--dump-http://localhost:8529/ourtest
HTTP/1.1200OK
content-type:text/plain
x-content-type-options:nosniff
"test"
Youwillseethatthemodule'sdofunctionhasbeenexecuted.
AWordaboutCachingSometimesitmightseemthatyourchangedonottakeeffect.Inthiscasetheculpritcouldbetheroutingcaches:
Theroutingcachestorestheroutinginformationcomputedfromthe_routingcollection.Wheneveryouchangethiscollectionmanually,youneedtocall
arangosh>require("internal").reloadRouting()
inordertorebuildthecache.
AdvancedUsagesFordetailedinformationseethereferencemanual.
Redirects
Usethefollowingforapermanentredirect:
arangosh>db._routing.save({
........>url:"/redirectMe",
........>action:{
........>do:"@arangodb/actions/redirectRequest",
........>options:{
Actions
652
........>permanently:true,
........>destination:"/somewhere.else/"
........>}
........>}
........>});
arangosh>require("internal").reloadRouting()
showexecutionresults
shell>curl--dump-http://localhost:8529/redirectMe
HTTP/1.1301MovedPermanently
x-content-type-options:nosniff
content-type:text/html
location:/somewhere.else/
"<html><head><title>Moved</title></head><body><h1>Moved</h1><p>Thispagehasmovedto<a
href=\"/somewhere.else/\">/somewhere.else/</a>.</p></body></html>"
RoutingBundles
Insteadofaddingallroutesforpackageseparately,youcanspecifyabundle:
arangosh>db._routing.save({
........>routes:[
........>{
........>url:"/url1",
........>content:"route1"
........>},
........>{
........>url:"/url2",
........>content:"route2"
........>},
........>{
........>url:"/url3",
........>content:"route3"
........>}
........>]
........>});
arangosh>require("internal").reloadRouting()
showexecutionresults
shell>curl--dump-http://localhost:8529/url2
HTTP/1.1200OK
content-type:text/plain
x-content-type-options:nosniff
"route2"
shell>curl--dump-http://localhost:8529/url3
HTTP/1.1200OK
content-type:text/plain
x-content-type-options:nosniff
Actions
653
"route3"
Theadvantageis,thatyoucanputallyourroutesintoonedocumentanduseacommonprefix.
arangosh>db._routing.save({
........>urlPrefix:"/test",
........>routes:[
........>{
........>url:"/url1",
........>content:"route1"
........>},
........>{
........>url:"/url2",
........>content:"route2"
........>},
........>{
........>url:"/url3",
........>content:"route3"
........>}
........>]
........>});
arangosh>require("internal").reloadRouting()
showexecutionresultswilldefinetheURL/test/url1,/test/url2,and/test/url3:
shell>curl--dump-http://localhost:8529/test/url1
HTTP/1.1200OK
content-type:text/plain
x-content-type-options:nosniff
"route1"
shell>curl--dump-http://localhost:8529/test/url2
HTTP/1.1200OK
content-type:text/plain
x-content-type-options:nosniff
"route2"
shell>curl--dump-http://localhost:8529/test/url3
HTTP/1.1200OK
content-type:text/plain
x-content-type-options:nosniff
"route3"
WritingMiddleware
Assume,youwanttologeveryrequestinyournamespacetotheconsole.(ifArangoDBisrunningasadaemon,thiswillendupinthelogfile).InthiscaseyoucaneasilydefineanactionfortheURL/subdirectory.Thisactionsimplylogstherequests,callsthenextinline,andlogstheresponse:
arangosh>db._modules.save({
Actions
654
........>path:"/db:/OwnMiddlewareTest",
........>content:
........>"exports.logRequest=function(req,res,options,next){"+
........>"console=require('console');"+
........>"console.log('receivedrequest:%s',JSON.stringify(req));"+
........>"next();"+
........>"console.log('producedresponse:%s',JSON.stringify(res));"+
........>"};"
........>});
showexecutionresultsThisfunctionwillnowbeavailableasdb://OwnMiddlewareTest/logRequest.YouneedtotellArangoDBthatitisshoulduseaprefixmatchandthattheshortestmatchshouldwininthiscase:
arangosh>db._routing.save({
........>middleware:[
........>{
........>url:{
........>match:"/subdirectory/*"
........>},
........>action:{
........>do:"db://OwnMiddlewareTest/logRequest"
........>}
........>}
........>]
........>});
showexecutionresultsWhenyoucallnext()inthataction,thenextspecificroutingwillbeusedfortheoriginalURL.EvenifyoumodifytheURLintherequestobjectreq,thiswillnotcausethenext()tojumptotheroutingdefinedforthisnextURL.IfproceedsoccurringtheoriginURL.However,ifyouusenext(true),theroutingwillstopandrequesthandlingisstartedwiththenewURL.Youmustensurethatnext(true)isnevercalledwithoutmodifyingtheURLintherequestobjectreq.Otherwiseanendlessloopwilloccur.
Nowweaddsomeothersimpleroutingstotestallthis:
arangosh>db._routing.save({
........>url:"/subdirectory/ourtest/1",
........>action:{
........>do:"@arangodb/actions/echoRequest"
........>}
........>});
arangosh>db._routing.save({
........>url:"/subdirectory/ourtest/2",
........>action:{
........>do:"@arangodb/actions/echoRequest"
........>}
........>});
arangosh>db._routing.save({
........>url:"/subdirectory/ourtest/3",
........>action:{
........>do:"@arangodb/actions/echoRequest"
........>}
........>});
arangosh>require("internal").reloadRouting()
showexecutionresults
Actions
655
Thenwesendsomecurlrequeststothesesampleroutes:
shell>curl--dump-http://localhost:8529/subdirectory/ourtest/1
HTTP/1.1200OK
content-type:application/json;charset=utf-8
x-content-type-options:nosniff
showresponsebodyandtheconsole(and/orthelogfile)willshowrequestsandreplies.Notethatloggingdoesn'twarrantthesequenceinwhichtheselineswillappear.
ApplicationDeploymentUsingsingleroutesorbundlescanbebecomeabitmessyinlargeapplications.Kaerushaswrittenadeploymenttoolinnode.js.
NotethatthereisalsoFoxxforbuildingapplicationswithArangoDB.
CommonPitfallswhenusingActions
Caching
IfyoumadeanychangestotheroutingbutthechangesdoesnothaveanyeffectwhencallingthemodifiedactionsURL,youmighthavebeenhitbysomecachingissues.
Afteranymodificationtotheroutingoractions,itisthusrecommendedtomakethechanges"live"bycallingthefollowingfunctionsfromwithinarangosh:
Youmightalsobeaffectedbyclient-sidecaching.BrowserstendtocachecontentandalsoredirectionURLs.Youmightneedtoclearordisablethebrowsercacheinsomecasestoseeyourchangesineffect.
Datatypes
Whenprocessingtherequestdatainanaction,pleasebeawarethatthedatatypeofallqueryparametersisstring.ThisisbecausethewholeURLisastringandwhentheindividualpartsareextracted,theywillalsobestrings.
Forexample,whencallingtheURLhttp://localhost:8529/hello/world?value=5
theparametervaluewillhaveavalueof(string)5,not(number)5.ThismightbetroublesomeifyouuseJavaScript's===operatorwhencheckingrequestparametervalues.
ThesameproblemoccurswithincomingHTTPheaders.WhensendingthefollowingheaderfromaclienttoArangoDB
X-My-Value:5
thentheheaderX-My-Valuewillhaveavalueof(string)5andnot(number)5.
404NotFound
IfyoudefinedaURLintheroutingandtheURLisaccessiblefineviaHTTPGETbutreturnsanHTTP501(notimplemented)forotherHTTPmethodssuchasPOST,PUTorDELETE,thenyoumighthavebeenhitbysomedefaults.
Bydefault,URLsdefinedlikethis(simplestringurlattribute)areaccessibleviaHTTPGETandHEADonly.TomakesuchURLsaccessibleviaotherHTTPmethods,extendtheURLdefinitionwiththemethodsattribute.
Forexample,thisdefinitiononlyallowsaccessviaGETandHEAD:
Actions
656
{
url:"/hello/world"
}
whereasthisdefinitionallowsHTTPGET,POST,andPUT:
arangosh>db._routing.save({
........>url:{
........>match:"/hello/world",
........>methods:["get","post","put"]
........>},
........>action:{
........>do:"@arangodb/actions/echoRequest"
........>}
........>});
arangosh>require("internal").reloadRouting()
showexecutionresults
shell>curl--dump-http://localhost:8529/hello/world
HTTP/1.1200OK
content-type:application/json;charset=utf-8
x-content-type-options:nosniff
showresponsebodyTheformerdefinition(definingurlasanobjectwithamatchattribute)willresultintheURLbeingaccessibleviaallsupportedHTTPmethods(e.g.GET,POST,PUT,DELETE,...),whereasthelatterdefinition(providingastringurlattribute)willresultintheURLbeingaccessibleviaHTTPGETandHTTPHEADonly,withallotherHTTPmethodsbeingdisabled.CallingaURLwithanunsupportedordisabledHTTPmethodwillresultinanHTTP404error.
Actions
657
Errorcodesandmeanings
Generalerrors
0-ERROR_NO_ERRORNoerrorhasoccurred.
1-ERROR_FAILEDWillberaisedwhenageneralerroroccurred.
2-ERROR_SYS_ERRORWillberaisedwhenoperatingsystemerroroccurred.
3-ERROR_OUT_OF_MEMORYWillberaisedwhenthereisamemoryshortage.
4-ERROR_INTERNALWillberaisedwhenaninternalerroroccurred.
5-ERROR_ILLEGAL_NUMBERWillberaisedwhenanillegalrepresentationofanumberwasgiven.
6-ERROR_NUMERIC_OVERFLOWWillberaisedwhenanumericoverflowoccurred.
7-ERROR_ILLEGAL_OPTIONWillberaisedwhenanunknownoptionwassuppliedbytheuser.
8-ERROR_DEAD_PIDWillberaisedwhenaPIDwithoutalivingprocesswasfound.
9-ERROR_NOT_IMPLEMENTEDWillberaisedwhenhittinganunimplementedfeature.
10-ERROR_BAD_PARAMETERWillberaisedwhentheparameterdoesnotfulfilltherequirements.
11-ERROR_FORBIDDENWillberaisedwhenyouaremissingpermissionfortheoperation.
12-ERROR_OUT_OF_MEMORY_MMAPWillberaisedwhenthereisamemoryshortage.
13-ERROR_CORRUPTED_CSVWillberaisedwhenencounteringacorruptcsvline.
14-ERROR_FILE_NOT_FOUNDWillberaisedwhenafileisnotfound.
15-ERROR_CANNOT_WRITE_FILEWillberaisedwhenafilecannotbewritten.
16-ERROR_CANNOT_OVERWRITE_FILEWillberaisedwhenanattemptismadetooverwriteanexistingfile.
17-ERROR_TYPE_ERRORWillberaisedwhenatypeerrorisunencountered.
18-ERROR_LOCK_TIMEOUTWillberaisedwhenthere'satimeoutwaitingforalock.
19-ERROR_CANNOT_CREATE_DIRECTORYWillberaisedwhenanattempttocreateadirectoryfails.
Errorcodesandmeanings
658
20-ERROR_CANNOT_CREATE_TEMP_FILEWillberaisedwhenanattempttocreateatemporaryfilefails.
21-ERROR_REQUEST_CANCELEDWillberaisedwhenarequestiscanceledbytheuser.
22-ERROR_DEBUGWillberaisedintentionallyduringdebugging.
25-ERROR_IP_ADDRESS_INVALIDWillberaisedwhenthestructureofanIPaddressisinvalid.
27-ERROR_FILE_EXISTSWillberaisedwhenafilealreadyexists.
28-ERROR_LOCKEDWillberaisedwhenaresourceoranoperationislocked.
29-ERROR_DEADLOCKWillberaisedwhenadeadlockisdetectedwhenaccessingcollections.
30-ERROR_SHUTTING_DOWNWillberaisedwhenacallcannotsucceedbecauseaservershutdownisalreadyinprogress.
31-ERROR_ONLY_ENTERPRISEWillberaisedwhenanenterprise-featureisrequestedfromthecommunityedition.
32-ERROR_RESOURCE_LIMITWillberaisedwhentheresourcesusedbyanoperationexceedtheconfiguredmaximumvalue.
HTTPerrorstatuscodes400-ERROR_HTTP_BAD_PARAMETERWillberaisedwhentheHTTPrequestdoesnotfulfilltherequirements.
401-ERROR_HTTP_UNAUTHORIZEDWillberaisedwhenauthorizationisrequiredbuttheuserisnotauthorized.
403-ERROR_HTTP_FORBIDDENWillberaisedwhentheoperationisforbidden.
404-ERROR_HTTP_NOT_FOUNDWillberaisedwhenanURIisunknown.
405-ERROR_HTTP_METHOD_NOT_ALLOWEDWillberaisedwhenanunsupportedHTTPmethodisusedforanoperation.
406-ERROR_HTTP_NOT_ACCEPTABLEWillberaisedwhenanunsupportedHTTPcontenttypeisusedforanoperation,orifarequestisnotacceptableforaleaderorfollower.
412-ERROR_HTTP_PRECONDITION_FAILEDWillberaisedwhenapreconditionforanHTTPrequestisnotmet.
500-ERROR_HTTP_SERVER_ERRORWillberaisedwhenaninternalserverisencountered.
503-ERROR_HTTP_SERVICE_UNAVAILABLEWillberaisedwhenaserviceistemporarilyunavailable.
HTTPprocessingerrors
Errorcodesandmeanings
659
600-ERROR_HTTP_CORRUPTED_JSONWillberaisedwhenastringrepresentationofaJSONobjectiscorrupt.
601-ERROR_HTTP_SUPERFLUOUS_SUFFICESWillberaisedwhentheURLcontainssuperfluoussuffices.
InternalArangoDBstorageerrors
Forerrorsthatoccurbecauseofaprogrammingerror.
1000-ERROR_ARANGO_ILLEGAL_STATEInternalerrorthatwillberaisedwhenthedatafileisnotintherequiredstate.
1002-ERROR_ARANGO_DATAFILE_SEALEDInternalerrorthatwillberaisedwhentryingtowritetoadatafile.
1004-ERROR_ARANGO_READ_ONLYInternalerrorthatwillberaisedwhentryingtowritetoaread-onlydatafileorcollection.
1005-ERROR_ARANGO_DUPLICATE_IDENTIFIERInternalerrorthatwillberaisedwhenaidentifierduplicateisdetected.
1006-ERROR_ARANGO_DATAFILE_UNREADABLEInternalerrorthatwillberaisedwhenadatafileisunreadable.
1007-ERROR_ARANGO_DATAFILE_EMPTYInternalerrorthatwillberaisedwhenadatafileisempty.
1008-ERROR_ARANGO_RECOVERYWillberaisedwhenanerroroccurredduringWALlogfilerecovery.
1009-ERROR_ARANGO_DATAFILE_STATISTICS_NOT_FOUNDWillberaisedwhenarequireddatafilestatisticsobjectwasnotfound.
ExternalArangoDBstorageerrorsForerrorsthatoccurbecauseofanoutsideevent.
1100-ERROR_ARANGO_CORRUPTED_DATAFILEWillberaisedwhenacorruptionisdetectedinadatafile.
1101-ERROR_ARANGO_ILLEGAL_PARAMETER_FILEWillberaisedifaparameterfileiscorruptedorcannotberead.
1102-ERROR_ARANGO_CORRUPTED_COLLECTIONWillberaisedwhenacollectioncontainsoneormorecorrupteddatafiles.
1103-ERROR_ARANGO_MMAP_FAILEDWillberaisedwhenthesystemcallmmapfailed.
1104-ERROR_ARANGO_FILESYSTEM_FULLWillberaisedwhenthefilesystemisfull.
1105-ERROR_ARANGO_NO_JOURNALWillberaisedwhenajournalcannotbecreated.
1106-ERROR_ARANGO_DATAFILE_ALREADY_EXISTSWillberaisedwhenthedatafilecannotbecreatedorrenamedbecauseafileofthesamenamealreadyexists.
1107-ERROR_ARANGO_DATADIR_LOCKEDWillberaisedwhenthedatabasedirectoryislockedbyadifferentprocess.
1108-ERROR_ARANGO_COLLECTION_DIRECTORY_ALREADY_EXISTSWillberaisedwhenthecollectioncannotbecreatedbecauseadirectoryofthesamenamealreadyexists.
Errorcodesandmeanings
660
1109-ERROR_ARANGO_MSYNC_FAILEDWillberaisedwhenthesystemcallmsyncfailed.
1110-ERROR_ARANGO_DATADIR_UNLOCKABLEWillberaisedwhentheservercannotlockthedatabasedirectoryonstartup.
1111-ERROR_ARANGO_SYNC_TIMEOUTWillberaisedwhentheserverwaitedtoolongforadatafiletobesyncedtodisk.
GeneralArangoDBstorageerrorsForerrorsthatoccurwhenfulfillingauserrequest.
1200-ERROR_ARANGO_CONFLICTWillberaisedwhenupdatingordeletingadocumentandaconflicthasbeendetected.
1201-ERROR_ARANGO_DATADIR_INVALIDWillberaisedwhenanon-existingdatabasedirectorywasspecifiedwhenstartingthedatabase.
1202-ERROR_ARANGO_DOCUMENT_NOT_FOUNDWillberaisedwhenadocumentwithagivenidentifierorhandleisunknown.
1203-ERROR_ARANGO_COLLECTION_NOT_FOUNDWillberaisedwhenacollectionwiththegivenidentifierornameisunknown.
1204-ERROR_ARANGO_COLLECTION_PARAMETER_MISSINGWillberaisedwhenthecollectionparameterismissing.
1205-ERROR_ARANGO_DOCUMENT_HANDLE_BADWillberaisedwhenadocumenthandleiscorrupt.
1206-ERROR_ARANGO_MAXIMAL_SIZE_TOO_SMALLWillberaisedwhenthemaximalsizeofthejournalistoosmall.
1207-ERROR_ARANGO_DUPLICATE_NAMEWillberaisedwhenanameduplicateisdetected.
1208-ERROR_ARANGO_ILLEGAL_NAMEWillberaisedwhenanillegalnameisdetected.
1209-ERROR_ARANGO_NO_INDEXWillberaisedwhennosuitableindexforthequeryisknown.
1210-ERROR_ARANGO_UNIQUE_CONSTRAINT_VIOLATEDWillberaisedwhenthereisauniqueconstraintviolation.
1211-ERROR_ARANGO_VIEW_NOT_FOUNDWillberaisedwhenaviewwiththegivenidentifierornameisunknown.
1212-ERROR_ARANGO_INDEX_NOT_FOUNDWillberaisedwhenanindexwithagivenidentifierisunknown.
1213-ERROR_ARANGO_CROSS_COLLECTION_REQUESTWillberaisedwhenacross-collectionisrequested.
1214-ERROR_ARANGO_INDEX_HANDLE_BADWillberaisedwhenaindexhandleiscorrupt.
1216-ERROR_ARANGO_DOCUMENT_TOO_LARGEWillberaisedwhenthedocumentcannotfitintoanydatafilebecauseofitistoolarge.
1217-ERROR_ARANGO_COLLECTION_NOT_UNLOADEDWillberaisedwhenacollectionshouldbeunloaded,buthasadifferentstatus.
Errorcodesandmeanings
661
1218-ERROR_ARANGO_COLLECTION_TYPE_INVALIDWillberaisedwhenaninvalidcollectiontypeisusedinarequest.
1219-ERROR_ARANGO_VALIDATION_FAILEDWillberaisedwhenthevalidationofanattributeofastructurefailed.
1220-ERROR_ARANGO_ATTRIBUTE_PARSER_FAILEDWillberaisedwhenparsinganattributenamedefinitionfailed.
1221-ERROR_ARANGO_DOCUMENT_KEY_BADWillberaisedwhenadocumentkeyiscorrupt.
1222-ERROR_ARANGO_DOCUMENT_KEY_UNEXPECTEDWillberaisedwhenauser-defineddocumentkeyissuppliedforcollectionswithautokeygeneration.
1224-ERROR_ARANGO_DATADIR_NOT_WRITABLEWillberaisedwhentheserver'sdatabasedirectoryisnotwritableforthecurrentuser.
1225-ERROR_ARANGO_OUT_OF_KEYSWillberaisedwhenakeygeneratorrunsoutofkeys.
1226-ERROR_ARANGO_DOCUMENT_KEY_MISSINGWillberaisedwhenadocumentkeyismissing.
1227-ERROR_ARANGO_DOCUMENT_TYPE_INVALIDWillberaisedwhenthereisanattempttocreateadocumentwithaninvalidtype.
1228-ERROR_ARANGO_DATABASE_NOT_FOUNDWillberaisedwhenanon-existingdatabaseisaccessed.
1229-ERROR_ARANGO_DATABASE_NAME_INVALIDWillberaisedwhenaninvaliddatabasenameisused.
1230-ERROR_ARANGO_USE_SYSTEM_DATABASEWillberaisedwhenanoperationisrequestedinadatabaseotherthanthesystemdatabase.
1231-ERROR_ARANGO_ENDPOINT_NOT_FOUNDWillberaisedwhenthereisanattempttodeleteanon-existingendpoint.
1232-ERROR_ARANGO_INVALID_KEY_GENERATORWillberaisedwhenaninvalidkeygeneratordescriptionisused.
1233-ERROR_ARANGO_INVALID_EDGE_ATTRIBUTEwillberaisedwhenthe_fromor_tovaluesofanedgeareundefinedorcontainaninvalidvalue.
1234-ERROR_ARANGO_INDEX_DOCUMENT_ATTRIBUTE_MISSINGWillberaisedwhenanattempttoinsertadocumentintoanindexiscausedbyinthedocumentnothavingoneormoreattributeswhichtheindexisbuilton.
1235-ERROR_ARANGO_INDEX_CREATION_FAILEDWillberaisedwhenanattempttocreateanindexhasfailed.
1236-ERROR_ARANGO_WRITE_THROTTLE_TIMEOUTWillberaisedwhentheserveriswrite-throttledandawriteoperationhaswaitedtoolongfortheservertoprocessqueuedoperations.
1237-ERROR_ARANGO_COLLECTION_TYPE_MISMATCHWillberaisedwhenacollectionhasadifferenttypefromwhathasbeenexpected.
1238-ERROR_ARANGO_COLLECTION_NOT_LOADEDWillberaisedwhenacollectionisaccessedthatisnotyetloaded.
1239-ERROR_ARANGO_DOCUMENT_REV_BADWillberaisedwhenadocumentrevisioniscorruptorismissingwhereneeded.
Errorcodesandmeanings
662
CheckedArangoDBstorageerrors
Forerrorsthatoccurbutareanticipated.
1300-ERROR_ARANGO_DATAFILE_FULLWillberaisedwhenthedatafilereachesitslimit.
1301-ERROR_ARANGO_EMPTY_DATADIRWillberaisedwhenencounteringanemptyserverdatabasedirectory.
1302-ERROR_ARANGO_TRY_AGAINWillberaisedwhenanoperationshouldberetried.
1303-ERROR_ARANGO_BUSYWillberaisedwhenstorageengineisbusy.
1304-ERROR_ARANGO_MERGE_IN_PROGRESSWillberaisedwhenstorageenginehasadatafilemergeinprogressandcannotcompletetheoperation.
1305-ERROR_ARANGO_IO_ERRORWillberaisedwhenstorageengineencountersanI/Oerror.
ArangoDBreplicationerrors1400-ERROR_REPLICATION_NO_RESPONSEWillberaisedwhenthereplicationapplierdoesnotreceiveanyoranincompleteresponsefromthemaster.
1401-ERROR_REPLICATION_INVALID_RESPONSEWillberaisedwhenthereplicationapplierreceivesaninvalidresponsefromthemaster.
1402-ERROR_REPLICATION_MASTER_ERRORWillberaisedwhenthereplicationapplierreceivesaservererrorfromthemaster.
1403-ERROR_REPLICATION_MASTER_INCOMPATIBLEWillberaisedwhenthereplicationapplierconnectstoamasterthathasanincompatibleversion.
1404-ERROR_REPLICATION_MASTER_CHANGEWillberaisedwhenthereplicationapplierconnectstoadifferentmasterthanbefore.
1405-ERROR_REPLICATION_LOOPWillberaisedwhenthereplicationapplierisaskedtoconnecttoitselfforreplication.
1406-ERROR_REPLICATION_UNEXPECTED_MARKERWillberaisedwhenanunexpectedmarkerisfoundinthereplicationlogstream.
1407-ERROR_REPLICATION_INVALID_APPLIER_STATEWillberaisedwhenaninvalidreplicationapplierstatefileisfound.
1408-ERROR_REPLICATION_UNEXPECTED_TRANSACTIONWillberaisedwhenanunexpectedtransactionidisfound.
1410-ERROR_REPLICATION_INVALID_APPLIER_CONFIGURATIONWillberaisedwhentheconfigurationforthereplicationapplierisinvalid.
1411-ERROR_REPLICATION_RUNNINGWillberaisedwhenthereisanattempttoperformanoperationwhilethereplicationapplierisrunning.
1412-ERROR_REPLICATION_APPLIER_STOPPEDSpecialerrorcodeusedtoindicatethereplicationapplierwasstoppedbyauser.
1413-ERROR_REPLICATION_NO_START_TICKWillberaisedwhenthereplicationapplierisstartedwithoutaknownstarttickvalue.
Errorcodesandmeanings
663
1414-ERROR_REPLICATION_START_TICK_NOT_PRESENTWillberaisedwhenthereplicationapplierfetchesdatausingastarttick,butthatstarttickisnotpresentontheloggerserveranymore.
ERROR_REPLICATION_WRONG_CHECKSUM_FORMAT,1415,"thechecksumformatiswrong","Willberaisedwhentheformatofthechecksumiswrong")
1416-ERROR_REPLICATION_WRONG_CHECKSUMWillberaisedwhenanewbornfollowersubmitsawrongchecksum
1417-ERROR_REPLICATION_SHARD_NONEMPTYWillberaisedwhenashardisnotemptyandthefollowertriesashortcut
ArangoDBclustererrors
1450-ERROR_CLUSTER_NO_AGENCYWillberaisedwhennoneoftheagencyserverscanbeconnectedto.
1451-ERROR_CLUSTER_NO_COORDINATOR_HEADERWillberaisedwhenaDBserverinaclusterreceivesaHTTPrequestwithoutacoordinatorheader.
1452-ERROR_CLUSTER_COULD_NOT_LOCK_PLANWillberaisedwhenacoordinatorinaclustercannotlockthePlanhierarchyintheagency.
1453-ERROR_CLUSTER_COLLECTION_ID_EXISTSWillberaisedwhenacoordinatorinaclustertriestocreateacollectionandthecollectionIDalreadyexists.
1454-ERROR_CLUSTER_COULD_NOT_CREATE_COLLECTION_IN_PLANWillberaisedwhenacoordinatorinaclustercannotcreateanentryforanewcollectioninthePlanhierarchyintheagency.
1455-ERROR_CLUSTER_COULD_NOT_READ_CURRENT_VERSIONWillberaisedwhenacoordinatorinaclustercannotreadtheVersionentryintheCurrenthierarchyintheagency.
1456-ERROR_CLUSTER_COULD_NOT_CREATE_COLLECTIONWillberaisedwhenacoordinatorinaclusternoticesthatsomeDBServersreportproblemswhencreatingshardsforanewcollection.
1457-ERROR_CLUSTER_TIMEOUTWillberaisedwhenacoordinatorinaclusterrunsintoatimeoutforsomeclusterwideoperation.
1458-ERROR_CLUSTER_COULD_NOT_REMOVE_COLLECTION_IN_PLANWillberaisedwhenacoordinatorinaclustercannotremoveanentryforacollectioninthePlanhierarchyintheagency.
1459-ERROR_CLUSTER_COULD_NOT_REMOVE_COLLECTION_IN_CURRENTWillberaisedwhenacoordinatorinaclustercannotremoveanentryforacollectionintheCurrenthierarchyintheagency.
1460-ERROR_CLUSTER_COULD_NOT_CREATE_DATABASE_IN_PLANWillberaisedwhenacoordinatorinaclustercannotcreateanentryforanewdatabaseinthePlanhierarchyintheagency.
1461-ERROR_CLUSTER_COULD_NOT_CREATE_DATABASEWillberaisedwhenacoordinatorinaclusternoticesthatsomeDBServersreportproblemswhencreatingdatabasesforanewclusterwidedatabase.
1462-ERROR_CLUSTER_COULD_NOT_REMOVE_DATABASE_IN_PLANWillberaisedwhenacoordinatorinaclustercannotremoveanentryforadatabaseinthePlanhierarchyintheagency.
1463-ERROR_CLUSTER_COULD_NOT_REMOVE_DATABASE_IN_CURRENTWillberaisedwhenacoordinatorinaclustercannotremoveanentryforadatabaseintheCurrenthierarchyintheagency.
1464-ERROR_CLUSTER_SHARD_GONEWillberaisedwhenacoordinatorinaclustercannotdeterminetheshardthatisresponsibleforagivendocument.
1465-ERROR_CLUSTER_CONNECTION_LOSTWillberaisedwhenacoordinatorinaclusterlosesanHTTPconnectiontoaDBserverintheclusterwhilsttransferringdata.
Errorcodesandmeanings
664
1466-ERROR_CLUSTER_MUST_NOT_SPECIFY_KEYWillberaisedwhenacoordinatorinaclusterfindsthatthe_keyattributewasspecifiedinashardedcollectiontheusesnotonly_keyasshardingattribute.
1467-ERROR_CLUSTER_GOT_CONTRADICTING_ANSWERSWillberaisedifacoordinatorinaclustergetsconflictingresultsfromdifferentshards,whichshouldneverhappen.
1468-ERROR_CLUSTER_NOT_ALL_SHARDING_ATTRIBUTES_GIVENWillberaisedifacoordinatortriestofindoutwhichshardisresponsibleforapartialdocument,butcannotdothisbecausenotallshardingattributesarespecified.
1469-ERROR_CLUSTER_MUST_NOT_CHANGE_SHARDING_ATTRIBUTESWillberaisedifthereisanattempttoupdatethevalueofashardattribute.
1470-ERROR_CLUSTER_UNSUPPORTEDWillberaisedwhenthereisanattempttocarryoutanoperationthatisnotsupportedinthecontextofashardedcollection.
1471-ERROR_CLUSTER_ONLY_ON_COORDINATORWillberaisedifthereisanattempttorunacoordinator-onlyoperationonadifferenttypeofnode.
1472-ERROR_CLUSTER_READING_PLAN_AGENCYWillberaisedifacoordinatororDBservercannotreadthePlanintheagency.
1473-ERROR_CLUSTER_COULD_NOT_TRUNCATE_COLLECTIONWillberaisedifacoordinatorcannottruncateallshardsofaclustercollection.
1474-ERROR_CLUSTER_AQL_COMMUNICATIONWillberaisediftheinternalcommunicationoftheclusterforAQLproducesanerror.
1475-ERROR_ARANGO_DOCUMENT_NOT_FOUND_OR_SHARDING_ATTRIBUTES_CHANGEDWillberaisedwhenadocumentwithagivenidentifierorhandleisunknown,oriftheshardingattributeshavebeenchangedinaREPLACEoperationinthecluster.
1476-ERROR_CLUSTER_COULD_NOT_DETERMINE_IDWillberaisedifaclusterserveratstartupcouldnotdetermineitsownIDfromthelocalinfoprovided.
1477-ERROR_CLUSTER_ONLY_ON_DBSERVERWillberaisedifthereisanattempttorunaDBserver-onlyoperationonadifferenttypeofnode.
1478-ERROR_CLUSTER_BACKEND_UNAVAILABLEWillberaisedifarequireddbservercan'tbereached.
1479-ERROR_CLUSTER_UNKNOWN_CALLBACK_ENDPOINTAnendpointcouldn'tbefound
1480-ERROR_CLUSTER_AGENCY_STRUCTURE_INVALIDThestructureintheagencyisinvalid
1481-ERROR_CLUSTER_AQL_COLLECTION_OUT_OF_SYNCWillberaisedifacollectionneededduringqueryexecutionisoutofsync.Thiscurrentlycanonlyhappenwhenusingsatellitecollections
1482-ERROR_CLUSTER_COULD_NOT_CREATE_INDEX_IN_PLANWillberaisedwhenacoordinatorinaclustercannotcreateanentryforanewindexinthePlanhierarchyintheagency.
1483-ERROR_CLUSTER_COULD_NOT_DROP_INDEX_IN_PLANWillberaisedwhenacoordinatorinaclustercannotremoveanindexfromthePlanhierarchyintheagency.
1484-ERROR_CLUSTER_CHAIN_OF_DISTRIBUTESHARDSLIKEWillberaisedifonetriestocreateacollectionwithadistributeShardsLikeattributewhichpointstoanothercollectionthatalsohasone.
1485-ERROR_CLUSTER_MUST_NOT_DROP_COLL_OTHER_DISTRIBUTESHARDSLIKEWillberaisedifonetriestodropacollectiontowhichanothercollectionpointswithitsdistributeShardsLikeattribute.
Errorcodesandmeanings
665
1486-ERROR_CLUSTER_UNKNOWN_DISTRIBUTESHARDSLIKEWillberaisedifonetriestocreateacollectionwhichpointstoanunknowncollectioninitsdistributeShardsLikeattribute.
1487-ERROR_CLUSTER_INSUFFICIENT_DBSERVERSWillberaisedifonetriestocreateacollectionwithareplicationFactorgreaterthantheavailablenumberofDBServers.
1488-ERROR_CLUSTER_COULD_NOT_DROP_FOLLOWERWillberaisedifafollowerthatoughttobedroppedcouldnotbedroppedintheagency(underCurrent).
1489-ERROR_CLUSTER_SHARD_LEADER_REFUSES_REPLICATIONWillberaisedifareplicationoperationisrefusedbyashardleader.
1490-ERROR_CLUSTER_SHARD_FOLLOWER_REFUSES_OPERATIONWillberaisedifanon-replicationoperationisrefusedbyashardfollower.
1491-ERROR_CLUSTER_SHARD_LEADER_RESIGNEDWillberaisedifanon-replicationoperationisrefusedbyaformershardleaderthathasfoundoutthatitisnolongertheleader.
1492-ERROR_CLUSTER_AGENCY_COMMUNICATION_FAILEDWillberaisedifaftervariousretriesanagencyoperationcouldnotbeperformedsuccessfully.
1493-ERROR_CLUSTER_DISTRIBUTE_SHARDS_LIKE_REPLICATION_FACTORWillberaisedifintendedreplicationfactordoesnotmatchthatoftheprototypeshardgiveninditributeShardsLikeparameter.
1494-ERROR_CLUSTER_DISTRIBUTE_SHARDS_LIKE_NUMBER_OF_SHARDSWillberaisedifintendednumberofshardsdoesnotmatchthatoftheprototypeshardgiveninditributeShardsLikeparameter.
1495-ERROR_CLUSTER_LEADERSHIP_CHALLENGE_ONGOINGWillberaisedwhenserversarecurrentlycompetingforleadership,andtheresultisstillunknown.
1496-ERROR_CLUSTER_NOT_LEADERWillberaisedwhenanoperationissenttoanon-leadingserver.
ArangoDBqueryerrors
1500-ERROR_QUERY_KILLEDWillberaisedwhenarunningqueryiskilledbyanexplicitadmincommand.
1501-ERROR_QUERY_PARSEWillberaisedwhenqueryisparsedandisfoundtobesyntacticallyinvalid.
1502-ERROR_QUERY_EMPTYWillberaisedwhenanemptyqueryisspecified.
1503-ERROR_QUERY_SCRIPTWillberaisedwhenaruntimeerroriscausedbythequery.
1504-ERROR_QUERY_NUMBER_OUT_OF_RANGEWillberaisedwhenanumberisoutsidetheexpectedrange.
1510-ERROR_QUERY_VARIABLE_NAME_INVALIDWillberaisedwhenaninvalidvariablenameisused.
1511-ERROR_QUERY_VARIABLE_REDECLAREDWillberaisedwhenavariablegetsre-assignedinaquery.
1512-ERROR_QUERY_VARIABLE_NAME_UNKNOWNWillberaisedwhenanunknownvariableisusedorthevariableisundefinedthecontextitisused.
1521-ERROR_QUERY_COLLECTION_LOCK_FAILEDWillberaisedwhenareadlockonthecollectioncannotbeacquired.
1522-ERROR_QUERY_TOO_MANY_COLLECTIONSWillberaisedwhenthenumberofcollectionsinaqueryisbeyondtheallowedvalue.
Errorcodesandmeanings
666
1530-ERROR_QUERY_DOCUMENT_ATTRIBUTE_REDECLAREDWillberaisedwhenadocumentattributeisre-assigned.
1540-ERROR_QUERY_FUNCTION_NAME_UNKNOWNWillberaisedwhenanundefinedfunctioniscalled.
1541-ERROR_QUERY_FUNCTION_ARGUMENT_NUMBER_MISMATCHWillberaisedwhenthenumberofargumentsusedinafunctioncalldoesnotmatchtheexpectednumberofargumentsforthefunction.
1542-ERROR_QUERY_FUNCTION_ARGUMENT_TYPE_MISMATCHWillberaisedwhenthetypeofanargumentusedinafunctioncalldoesnotmatchtheexpectedargumenttype.
1543-ERROR_QUERY_INVALID_REGEXWillberaisedwhenaninvalidregexargumentvalueisusedinacalltoafunctionthatexpectsaregex.
1550-ERROR_QUERY_BIND_PARAMETERS_INVALIDWillberaisedwhenthestructureofbindparameterspassedhasanunexpectedformat.
1551-ERROR_QUERY_BIND_PARAMETER_MISSINGWillberaisedwhenabindparameterwasdeclaredinthequerybutthequeryisbeingexecutedwithnovalueforthatparameter.
1552-ERROR_QUERY_BIND_PARAMETER_UNDECLAREDWillberaisedwhenavaluegetsspecifiedforanundeclaredbindparameter.
1553-ERROR_QUERY_BIND_PARAMETER_TYPEWillberaisedwhenabindparameterhasaninvalidvalueortype.
1560-ERROR_QUERY_INVALID_LOGICAL_VALUEWillberaisedwhenanon-booleanvalueisusedinalogicaloperation.
1561-ERROR_QUERY_INVALID_ARITHMETIC_VALUEWillberaisedwhenanon-numericvalueisusedinanarithmeticoperation.
1562-ERROR_QUERY_DIVISION_BY_ZEROWillberaisedwhenthereisanattempttodividebyzero.
1563-ERROR_QUERY_ARRAY_EXPECTEDWillberaisedwhenanon-arrayoperandisusedforanoperationthatexpectsanarrayargumentoperand.
1569-ERROR_QUERY_FAIL_CALLEDWillberaisedwhenthefunctionFAIL()iscalledfrominsideaquery.
1570-ERROR_QUERY_GEO_INDEX_MISSINGWillberaisedwhenageorestrictionwasspecifiedbutnosuitablegeoindexisfoundtoresolveit.
1571-ERROR_QUERY_FULLTEXT_INDEX_MISSINGWillberaisedwhenafulltextqueryisperformedonacollectionwithoutasuitablefulltextindex.
1572-ERROR_QUERY_INVALID_DATE_VALUEWillberaisedwhenavaluecannotbeconvertedtoadate.
1573-ERROR_QUERY_MULTI_MODIFYWillberaisedwhenanAQLquerycontainsmorethanonedata-modifyingoperation.
1574-ERROR_QUERY_INVALID_AGGREGATE_EXPRESSIONWillberaisedwhenanAQLquerycontainsaninvalidaggregateexpression.
1575-ERROR_QUERY_COMPILE_TIME_OPTIONSWillberaisedwhenanAQLdata-modificationquerycontainsoptionsthatcannotbefiguredoutatquerycompiletime.
1576-ERROR_QUERY_EXCEPTION_OPTIONSWillberaisedwhenanAQLdata-modificationquerycontainsaninvalidoptionsspecification.
1577-ERROR_QUERY_COLLECTION_USED_IN_EXPRESSIONWillberaisedwhenacollectionisusedasanoperandinanAQLexpression.
Errorcodesandmeanings
667
1578-ERROR_QUERY_DISALLOWED_DYNAMIC_CALLWillberaisedwhenadynamicfunctioncallismadetoafunctionthatcannotbecalleddynamically.
1579-ERROR_QUERY_ACCESS_AFTER_MODIFICATIONWillberaisedwhencollectiondataareaccessedafteradata-modificationoperation.
AQLuserfunctionerrors
1580-ERROR_QUERY_FUNCTION_INVALID_NAMEWillberaisedwhenauserfunctionwithaninvalidnameisregistered.
1581-ERROR_QUERY_FUNCTION_INVALID_CODEWillberaisedwhenauserfunctionisregisteredwithinvalidcode.
1582-ERROR_QUERY_FUNCTION_NOT_FOUNDWillberaisedwhenauserfunctionisaccessedbutnotfound.
1583-ERROR_QUERY_FUNCTION_RUNTIME_ERRORWillberaisedwhenauserfunctionthrowsaruntimeexception.
AQLqueryregistryerrors1590-ERROR_QUERY_BAD_JSON_PLANWillberaisedwhenanHTTPAPIforaquerygotaninvalidJSONobject.
1591-ERROR_QUERY_NOT_FOUNDWillberaisedwhenanIdofaqueryisnotfoundbytheHTTPAPI.
1592-ERROR_QUERY_IN_USEWillberaisedwhenanIdofaqueryisfoundbytheHTTPAPIbutthequeryisinuse.
ArangoDBcursorerrors
1600-ERROR_CURSOR_NOT_FOUNDWillberaisedwhenacursorisrequestedviaitsidbutacursorwiththatidcannotbefound.
1601-ERROR_CURSOR_BUSYWillberaisedwhenacursorisrequestedviaitsidbutaconcurrentrequestisstillusingthecursor.
ArangoDBtransactionerrors1650-ERROR_TRANSACTION_INTERNALWillberaisedwhenawrongusageoftransactionsisdetected.thisisaninternalerrorandindicatesabuginArangoDB.
1651-ERROR_TRANSACTION_NESTEDWillberaisedwhentransactionsarenested.
1652-ERROR_TRANSACTION_UNREGISTERED_COLLECTIONWillberaisedwhenacollectionisusedinthemiddleofatransactionbutwasnotregisteredattransactionstart.
1653-ERROR_TRANSACTION_DISALLOWED_OPERATIONWillberaisedwhenadisallowedoperationiscarriedoutinatransaction.
1654-ERROR_TRANSACTION_ABORTEDWillberaisedwhenatransactionwasaborted.
Usermanagementerrors
Errorcodesandmeanings
668
1700-ERROR_USER_INVALID_NAMEWillberaisedwhenaninvalidusernameisused.
1701-ERROR_USER_INVALID_PASSWORDWillberaisedwhenaninvalidpasswordisused.
1702-ERROR_USER_DUPLICATEWillberaisedwhenausernamealreadyexists.
1703-ERROR_USER_NOT_FOUNDWillberaisedwhenausernameisupdatedthatdoesnotexist.
1704-ERROR_USER_CHANGE_PASSWORDWillberaisedwhentheusermustchangehispassword.
1705-ERROR_USER_EXTERNALWillberaisedwhentheuserisauthenicatedbyanexternalserver.
Servicemanagementerrors(legacy)
ThesehavebeensupercededbytheFoxxmanagementerrorsinpublicAPIs.
1750-ERROR_SERVICE_INVALID_NAMEWillberaisedwhenaninvalidservicenameisspecified.
1751-ERROR_SERVICE_INVALID_MOUNTWillberaisedwhenaninvalidmountisspecified.
1752-ERROR_SERVICE_DOWNLOAD_FAILEDWillberaisedwhenaservicedownloadfromthecentralrepositoryfailed.
1753-ERROR_SERVICE_UPLOAD_FAILEDWillberaisedwhenaserviceuploadfromtheclienttotheArangoDBserverfailed.
LDAPerrors1800-ERROR_LDAP_CANNOT_INITcannotinitaLDAPconnection
1801-ERROR_LDAP_CANNOT_SET_OPTIONcannotsetaLDAPoption
1802-ERROR_LDAP_CANNOT_BINDcannotbindtoaLDAPserver
1803-ERROR_LDAP_CANNOT_UNBINDcannotunbindfromaLDAPserver
1804-ERROR_LDAP_CANNOT_SEARCHcannotsearchtheLDAPserver
1805-ERROR_LDAP_CANNOT_START_TLScannotstaraTLSLDAPsession
1806-ERROR_LDAP_FOUND_NO_OBJECTSLDAPdidn'tfoundanyobjectswiththespecifiedsearchquery
1807-ERROR_LDAP_NOT_ONE_USER_FOUNDLDAPfoundzerooremorethanoneuser
1808-ERROR_LDAP_USER_NOT_IDENTIFIEDLDAPfoundauser,butitsnotthedesiredone
Errorcodesandmeanings
669
1820-ERROR_LDAP_INVALID_MODEcantdistinguishavalidmodeforprovidedldapconfiguration
Taskerrors
1850-ERROR_TASK_INVALID_IDWillberaisedwhenataskiscreatedwithaninvalidid.
1851-ERROR_TASK_DUPLICATE_IDWillberaisedwhenataskidiscreatedwithaduplicateid.
1852-ERROR_TASK_NOT_FOUNDWillberaisedwhenataskwiththespecifiedidcouldnotbefound.
Graph/traversalerrors1901-ERROR_GRAPH_INVALID_GRAPHWillberaisedwhenaninvalidnameispassedtotheserver.
1902-ERROR_GRAPH_COULD_NOT_CREATE_GRAPHWillberaisedwhenaninvalidname,verticesoredgesispassedtotheserver.
1903-ERROR_GRAPH_INVALID_VERTEXWillberaisedwhenaninvalidvertexidispassedtotheserver.
1904-ERROR_GRAPH_COULD_NOT_CREATE_VERTEXWillberaisedwhenthevertexcouldnotbecreated.
1905-ERROR_GRAPH_COULD_NOT_CHANGE_VERTEXWillberaisedwhenthevertexcouldnotbechanged.
1906-ERROR_GRAPH_INVALID_EDGEWillberaisedwhenaninvalidedgeidispassedtotheserver.
1907-ERROR_GRAPH_COULD_NOT_CREATE_EDGEWillberaisedwhentheedgecouldnotbecreated.
1908-ERROR_GRAPH_COULD_NOT_CHANGE_EDGEWillberaisedwhentheedgecouldnotbechanged.
1909-ERROR_GRAPH_TOO_MANY_ITERATIONSWillberaisedwhentoomanyiterationsaredoneinagraphtraversal.
1910-ERROR_GRAPH_INVALID_FILTER_RESULTWillberaisedwhenaninvalidfilterresultisreturnedinagraphtraversal.
1920-ERROR_GRAPH_COLLECTION_MULTI_USEanedgecollectionmayonlybeusedonceinoneedgedefinitionofagraph.,
1921-ERROR_GRAPH_COLLECTION_USE_IN_MULTI_GRAPHSisalreadyusedbyanothergraphinadifferentedgedefinition.,
1922-ERROR_GRAPH_CREATE_MISSING_NAMEagraphnameisrequiredtocreateagraph.,
1923-ERROR_GRAPH_CREATE_MALFORMED_EDGE_DEFINITIONtheedgedefinitionismalformed.Ithastobeanarrayofobjects.,
1924-ERROR_GRAPH_NOT_FOUNDagraphwiththisnamecouldnotbefound.,
1925-ERROR_GRAPH_DUPLICATEagraphwiththisnamealreadyexists.,
Errorcodesandmeanings
670
1926-ERROR_GRAPH_VERTEX_COL_DOES_NOT_EXISTthespecifiedvertexcollectiondoesnotexistorisnotpartofthegraph.,
1927-ERROR_GRAPH_WRONG_COLLECTION_TYPE_VERTEXthecollectionisnotavertexcollection.,
1928-ERROR_GRAPH_NOT_IN_ORPHAN_COLLECTIONVertexcollectionnotinorphancollectionofthegraph.,
1929-ERROR_GRAPH_COLLECTION_USED_IN_EDGE_DEFThecollectionisalreadyusedinanedgedefinitionofthegraph.,
1930-ERROR_GRAPH_EDGE_COLLECTION_NOT_USEDTheedgecollectionisnotusedinanyedgedefinitionofthegraph.,
1931-ERROR_GRAPH_NOT_AN_ARANGO_COLLECTIONThecollectionisnotanArangoCollection.,
1932-ERROR_GRAPH_NO_GRAPH_COLLECTIONcollection_graphsdoesnotexist.,
1933-ERROR_GRAPH_INVALID_EXAMPLE_ARRAY_OBJECT_STRINGInvalidexampletype.HastobeString,ArrayorObject.,
1934-ERROR_GRAPH_INVALID_EXAMPLE_ARRAY_OBJECTInvalidexampletype.HastobeArrayorObject.,
1935-ERROR_GRAPH_INVALID_NUMBER_OF_ARGUMENTSInvalidnumberofarguments.Expected:,
1936-ERROR_GRAPH_INVALID_PARAMETERInvalidparametertype.,
1937-ERROR_GRAPH_INVALID_IDInvalidid,
1938-ERROR_GRAPH_COLLECTION_USED_IN_ORPHANSThecollectionisalreadyusedintheorphansofthegraph.,
1939-ERROR_GRAPH_EDGE_COL_DOES_NOT_EXISTthespecifiededgecollectiondoesnotexistorisnotpartofthegraph.,
1940-ERROR_GRAPH_EMPTYTherequestedgraphhasnoedgecollections.
Sessionerrors1950-ERROR_SESSION_UNKNOWNWillberaisedwhenaninvalid/unknownsessionidispassedtotheserver.
1951-ERROR_SESSION_EXPIREDWillberaisedwhenasessionisexpired.
SimpleClienterrors
2000-SIMPLE_CLIENT_UNKNOWN_ERRORThiserrorshouldnothappen.
2001-SIMPLE_CLIENT_COULD_NOT_CONNECTWillberaisedwhentheclientcouldnotconnecttotheserver.
2002-SIMPLE_CLIENT_COULD_NOT_WRITEWillberaisedwhentheclientcouldnotwritedata.
Errorcodesandmeanings
671
2003-SIMPLE_CLIENT_COULD_NOT_READWillberaisedwhentheclientcouldnotreaddata.
Communicatorerrors
2100-COMMUNICATOR_REQUEST_ABORTEDRequestwasaborted.
2101-COMMUNICATOR_DISABLEDCommunicationwasdisabled.
Foxxmanagementerrors
3000-ERROR_MALFORMED_MANIFEST_FILETheservicemanifestfileisnotwell-formedJSON.
3001-ERROR_INVALID_SERVICE_MANIFESTTheservicemanifestcontainsinvalidvalues.
3002-ERROR_SERVICE_FILES_MISSINGTheservicefolderorbundledoesnotexistonthisserver.
3003-ERROR_SERVICE_FILES_OUTDATEDThelocalservicebundledoesnotmatchthechecksuminthedatabase.
3004-ERROR_INVALID_FOXX_OPTIONSTheserviceoptionscontaininvalidvalues.
3007-ERROR_INVALID_MOUNTPOINTTheservicemountpathcontainsinvalidcharacters.
3009-ERROR_SERVICE_NOT_FOUNDNoservicefoundatthegivenmountpath.
3010-ERROR_SERVICE_NEEDS_CONFIGURATIONTheserviceismissingconfigurationordependencies.
3011-ERROR_SERVICE_MOUNTPOINT_CONFLICTAservicealreadyexistsatthegivenmountpath.
3012-ERROR_SERVICE_MANIFEST_NOT_FOUNDTheservicedirectorydoesnotcontainamanifestfile.
3013-ERROR_SERVICE_OPTIONS_MALFORMEDTheserviceoptionsarenotwell-formedJSON.
3014-ERROR_SERVICE_SOURCE_NOT_FOUNDThesourcepathdoesnotmatchafileordirectory.
3015-ERROR_SERVICE_SOURCE_ERRORThesourcepathcouldnotberesolved.
3016-ERROR_SERVICE_UNKNOWN_SCRIPTTheservicedoesnothaveascriptwiththisname.
JavaScriptmoduleloadererrors3100-ERROR_MODULE_NOT_FOUNDThemodulepathcouldnotberesolved.
Errorcodesandmeanings
672
3101-ERROR_MODULE_SYNTAX_ERRORThemodulecouldnotbeparsedbecauseofasyntaxerror.
3103-ERROR_MODULE_FAILUREFailedtoinvokethemoduleinitscontext.
Enterpriseerrors
4000-ERROR_NO_SMART_COLLECTIONTherequestedcollectionneedstobesmart,butitain't
4001-ERROR_NO_SMART_GRAPH_ATTRIBUTEThegivendocumentdoesnothavethesmartgraphattributeset.
4002-ERROR_CANNOT_DROP_SMART_COLLECTIONThissmartcollectioncannotbedropped,itdictatesshardinginthegraph.
4003-ERROR_KEY_MUST_BE_PREFIXED_WITH_SMART_GRAPH_ATTRIBUTEInasmartvertexcollection_keymustbeprefixedwiththevalueofthesmartgraphattribute.
4004-ERROR_ILLEGAL_SMART_GRAPH_ATTRIBUTEThegivensmartGraphattributeisillegalandconnotbeusedforsharding.Allsystemattributesareforbidden.
Agencyerrors
20001-ERROR_AGENCY_INQUIRY_SYNTAXInquiryhandlesalistofstringclientIds:[,...].
20011-ERROR_AGENCY_INFORM_MUST_BE_OBJECTTheinformmessageintheagencymustbeanobject.
20012-ERROR_AGENCY_INFORM_MUST_CONTAIN_TERMTheinformmessageintheagencymustcontainauintparameter'term'.
20013-ERROR_AGENCY_INFORM_MUST_CONTAIN_IDTheinformmessageintheagencymustcontainastringparameter'id'.
20014-ERROR_AGENCY_INFORM_MUST_CONTAIN_ACTIVETheinformmessageintheagencymustcontainanarray'active'.
20015-ERROR_AGENCY_INFORM_MUST_CONTAIN_POOLTheinformmessageintheagencymustcontainanobject'pool'.
20016-ERROR_AGENCY_INFORM_MUST_CONTAIN_MIN_PINGTheinformmessageintheagencymustcontainanobject'minping'.
20017-ERROR_AGENCY_INFORM_MUST_CONTAIN_MAX_PINGTheinformmessageintheagencymustcontainanobject'maxping'.
20018-ERROR_AGENCY_INFORM_MUST_CONTAIN_TIMEOUT_MULTTheinformmessageintheagencymustcontainanobject'timeoutMult'.
20020-ERROR_AGENCY_INQUIRE_CLIENT_ID_MUST_BE_STRINGInquirybyclientIdfailed
20021-ERROR_AGENCY_CANNOT_REBUILD_DBSWillberaisedifthereadDBorthespearHeadcannotberebuiltfromthereplicatedlog.
Supervisionerrors
20501-ERROR_SUPERVISION_GENERAL_FAILURE
Errorcodesandmeanings
673
Generalsupervisionfailure.
Dispatchererrors
21001-ERROR_DISPATCHER_IS_STOPPINGWillbereturnedifashutdownisinprogress.
21002-ERROR_QUEUE_UNKNOWNWillbereturnedifaqueuewiththisnamedoesnotexist.
21003-ERROR_QUEUE_FULLWillbereturnedifaqueuewiththisnameisfull.
Errorcodesandmeanings
674
Glossary
Collection
Acollectionconsistsofdocuments.Itisuniquelyidentifiedbyitscollectionidentifier.Italsohasauniquenamethatclientsshouldusetoidentifyandaccessit.Collectionscanberenamed.Itwillchangethecollectionname,butnotthecollectionidentifier.Collectionscontaindocumentsofaspecifictype.Therearecurrentlytwotypes:document(default)andedge.Thetypeisspecifiedbytheuserwhenthecollectioniscreated,andcannotbechangedlater.
CollectionIdentifier
Acollectionidentifieridentifiesacollectioninadatabase.Itisastringvalueandisuniquewithinthedatabase.UptoincludingArangoDB1.1,thecollectionidentifierhasbeenaclient'sprimarymeanstoaccesscollections.StartingwithArangoDB1.2,clientsshouldinsteaduseacollection'suniquenametoaccessacollectioninsteadofitsidentifier.
ArangoDBcurrentlyuses64bitunsignedintegervaluestomaintaincollectionidsinternally.Whenreturningcollectionidstoclients,ArangoDBwillputthemintoastringtoensurethecollectionidisnotclippedbyclientsthatdonotsupportbigintegers.ClientsshouldtreatthecollectionidsreturnedbyArangoDBasopaquestringswhentheystoreoruseitlocally.
CollectionName
Acollectionnameidentifiesacollectioninadatabase.Itisastringandisuniquewithinthedatabase.Unlikethecollectionidentifieritissuppliedbythecreatorofthecollection.Thecollectionnamemustconsistofletters,digits,andthe_(underscore)and-(dash)charactersonly.PleaserefertoNamingConventionsformoreinformationonvalidcollectionnames.
Database
ArangoDBcanhandlemultipledatabasesinthesameserverinstance.Databasescanbeusedtologicallygroupandseparatedata.AnArangoDBdatabaseconsistsofcollectionsanddedicateddatabase-specificworkerprocesses.
Adatabasecontainsitsowncollections(whichcannotbeaccessedfromotherdatabases),Foxxapplications,andreplicationloggersandappliers.EachArangoDBdatabasecontainsitsownsystemcollections(e.g._users,_replication,...).
TherewillalwaysbeatleastonedatabaseinArangoDB.Thisisthedefaultdatabase,named_system.Thisdatabasecannotbedropped,andprovidesspecialoperationsforcreating,dropping,andenumeratingdatabases.Userscancreateadditionaldatabasesandgivethemuniquenamestoaccessthemlater.Databasemanagementoperationscannotbeinitiatedfromoutofuser-defineddatabases.
WhenArangoDBisaccessedviaitsHTTPRESTAPI,thedatabasenameisreadfromthefirstpartoftherequestURIpath(e.g./_db/_system/...).IftherequestURIdoesnotcontainadatabasename,thedatabasenameisautomaticallyderivedfromtheendpoint.PleaserefertoDatabaseEndpointformoreinformation.
DatabaseName
AsingleArangoDBinstancecanhandlemultipledatabasesinparallel.Whenmultipledatabasesareused,eachdatabasemustbegivenauniquename.Thisnameisusedtouniquelyidentifyadatabase.ThedefaultdatabaseinArangoDBisnamed_system.
Thedatabasenameisastringconsistingofonlyletters,digitsandthe_(underscore)and-(dash)characters.User-defineddatabasenamesmustalwaysstartwithaletter.Databasenamesiscase-sensitive.
DatabaseOrganization
AsingleArangoDBinstancecanhandlemultipledatabasesinparallel.Bydefault,therewillbeatleastonedatabase,whichisnamed_system.
Databasesarephysicallystoredinseparatesub-directoriesunderneaththedatabasedirectory,whichitselfresidesintheinstance'sdatadirectory.
Glossary
675
Eachdatabasehasitsownsub-directory,nameddatabase-.Thedatabasedirectorycontainssub-directoriesforthecollectionsofthedatabase,andafilenamedparameter.json.Thisfilecontainsthedatabaseidandname.
InanexampleArangoDBinstancewhichhastwodatabases,thefilesystemlayoutcouldlooklikethis:
data/#theinstance'sdatadirectory
databases/#sub-directorycontainingalldatabases'data
database-<id>/#sub-directoryforasingledatabase
parameter.json#filecontainingdatabaseidandname
collection-<id>/#directorycontainingdataaboutacollection
database-<id>/#sub-directoryforanotherdatabase
parameter.json#filecontainingdatabaseidandname
collection-<id>/#directorycontainingdataaboutacollection
collection-<id>/#directorycontainingdataaboutacollection
Foxxapplicationsarealsoorganizedindatabase-specificdirectoriesinsidetheapplicationpath.Thefilesystemlayoutcouldlooklikethis:
apps/#theinstance'sapplicationdirectory
system/#systemapplications(canbeignored)
_db/#sub-directorycontainingdatabase-specificapplications
<database-name>/#sub-directoryforasingledatabase
<mountpoint>/APP#sub-directoryforasingleapplication
<mountpoint>/APP#sub-directoryforasingleapplication
<database-name>/#sub-directoryforanotherdatabase
<mountpoint>/APP#sub-directoryforasingleapplication
Document
DocumentsinArangoDBareJSONobjects.Theseobjectscanbenested(toanydepth)andmaycontainarrays.Eachdocumentisuniquelyidentifiedbyitsdocumenthandle.
DocumentEtag
Thedocumentrevision(_revvalue)enclosedindoublequotes.TherevisionisreturnedbyseveralHTTPAPImethodsintheEtagHTTPheader.
DocumentHandle
Adocumenthandleuniquelyidentifiesadocumentinthedatabase.Itisastringandconsistsofthecollection'snameandthedocumentkey(_keyattribute)separatedby/.Thedocumenthandleisstoredinadocument's_idattribute.
DocumentKey
Adocumentkeyisastringthatuniquelyidentifiesadocumentinagivencollection.Itcanandshouldbeusedbyclientswhenspecificdocumentsaresearched.Documentkeysarestoredinthe_keyattributeofdocuments.ThekeyvaluesareautomaticallyindexedbyArangoDBinacollection'sprimaryindex.Thuslookingupadocumentbyitskeyisregularlyafastoperation.The_keyvalueofadocumentisimmutableoncethedocumenthasbeencreated.
Bydefault,ArangoDBwillauto-generateadocumentkeyifno_keyattributeisspecified,andusetheuser-specified_keyvalueotherwise.
Thisbehaviorcanbechangedonaper-collectionlevelbycreatingcollectionswiththekeyOptionsattribute.
UsingkeyOptionsitispossibletodisallowuser-specifiedkeyscompletely,ortoforceaspecificregimeforauto-generatingthe_keyvalues.
Therearesomerestrictionsforuser-definedkeys(seeNamingConventionsfordocumentkeys).
DocumentRevision
Glossary
676
AsArangoDBsupportsMVCC,documentscanexistinmorethanonerevision.ThedocumentrevisionistheMVCCtokenusedtoidentifyaparticularrevisionofadocument.Itisastringvaluecurrentlycontaininganintegernumberandisuniquewithinthelistofdocumentrevisionsforasingledocument.Documentrevisionscanbeusedtoconditionallyupdate,replaceordeletedocumentsinthedatabase.Inordertofindaparticularrevisionofadocument,youneedthedocumenthandleandthedocumentrevision.
Thedocumentrevisionisstoredinthe_revattributeofadocument,andissetandupdatedbyArangoDBautomatically.The_revvaluecannotbesetfromtheoutside.
ArangoDBcurrentlyuses64bitunsignedintegervaluestomaintaindocumentrevisionsinternally.Whenreturningdocumentrevisionstoclients,ArangoDBwillputthemintoastringtoensuretherevisionidisnotclippedbyclientsthatdonotsupportbigintegers.ClientsshouldtreattherevisionidreturnedbyArangoDBasanopaquestringwhentheystoreoruseitlocally.ThiswillallowArangoDBtochangetheformatofrevisionidslaterifthisshouldberequired.Clientscanuserevisionsidstoperformsimpleequality/non-equalitycomparisons(e.g.tocheckwhetheradocumenthaschangedornot),buttheyshouldnotuserevisionidstoperformgreater/lessthancomparisonswiththemtocheckifadocumentrevisionisolderthanoneanother,evenifthismightworkforsomecases.
Edge
Edgesarespecialdocumentsusedforconnectingotherdocumentsintoagraph.Anedgedescribestheconnectionbetweentwodocumentsusingtheinternalattributes:_fromand_to.Thesecontaindocumenthandles,namelythestart-pointandtheend-pointoftheedge.
EdgeCollection
Edgecollectionsarecollectionsthatstoreedges.
EdgeDefinition
Edgedefinitionsarepartsofthedefinitionofnamedgraphs.Theydescribewhichedgecollectionsconnectwhichvertexcollections.
GeneralGraph
Modulemaintaininggraphsetupinthe_graphscollection-akanamedgraphs.Configureswhichedgecollectionsrelatetowhichvertexcollections.Ensuresgraphconsistencyinmodificationqueries.
NamedGraphs
Namedgraphsenforceconsistencybetweenedgecollectionsandvertexcollections,soifyouremoveavertex,edgespointingtoitwillberemovedtoo.
Index
Indexesareusedtoallowfastaccesstodocumentsinacollection.Allcollectionshaveaprimaryindex,whichisthedocument's_keyattribute.Thisindexcannotbedroppedorchanged.
Edgecollectionswillalsohaveanautomaticallycreatededgesindex,whichcannotbemodified.Thisindexprovidesquickaccesstodocumentsviathe_fromand_toattributes.
Mostuser-landindexescanbecreatedbydefiningthenamesoftheattributeswhichshouldbeindexed.Someindextypesallowindexingjustoneattribute(e.g.fulltextindex)whereasotherindextypesallowindexingmultipleattributes.
Indexingthesystemattribute_idinuser-definedindexesisnotsupportedbyanyindextype.
EdgesIndex
Anedgesindexisautomaticallycreatedforedgecollections.Itcontainsconnectionsbetweenvertexdocumentsandisinvokedwhentheconnectingedgesofavertexarequeried.Thereisnowaytoexplicitlycreateordeleteedgesindexes.
FulltextIndex
Glossary
677
Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.Afulltextindexcanbedefinedononeattributeonly,andwillincludeallwordscontainedindocumentsthathaveatextualvalueintheindexattribute.SinceArangoDB2.6theindexwillalsoincludewordsfromtheindexattributeiftheindexattributeisanarrayofstrings,oranobjectwithstringvaluemembers.
Forexample,givenafulltextindexonthetranslationsattributeandthefollowingdocuments,thensearchingforлисаusingthefulltextindexwouldreturnonlythefirstdocument.SearchingfortheindexfortheexactstringFoxwouldreturnthefirsttwodocuments,andsearchingforprefix:Foxwouldreturnallthreedocuments:
{translations:{en:"fox",de:"Fuchs",fr:"renard",ru:"лиса"}}
{translations:"FoxistheEnglishtranslationoftheGermanwordFuchs"}
{translations:["ArangoDB","document","database","Foxx"]}
Iftheindexattributeisneitherastring,anobjectoranarray,itscontentswillnotbeindexed.Whenindexingthecontentsofanarrayattribute,anarraymemberwillonlybeincludedintheindexifitisastring.Whenindexingthecontentsofanobjectattribute,anobjectmembervaluewillonlybeincludedintheindexifitisastring.Otherdatatypesareignoredandnotindexed.
Onlywordswitha(specifiable)minimumlengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedbylibicu,whichistakingintoaccounttheselectedlanguageprovidedatserverstart.Wordsareindexedintheirlower-casedform.Theindexsupportscompletematchqueries(fullwords)andprefixqueries.
GeoIndex
Ageoindexisusedtofindplacesonthesurfaceoftheearthfast.
IndexHandle
Anindexhandleuniquelyidentifiesanindexinthedatabase.Itisastringandconsistsofacollectionnameandanindexidentifierseparatedby/.
HashIndex
Ahashindexisusedtofinddocumentsbasedonexamples.Ahashindexcanbecreatedforoneormultipledocumentattributes.
Ahashindexwillonlybeusedbyqueriesifallindexedattributesarepresentintheexampleorsearchquery,andifallattributesarecomparedusingtheequality(==operator).Thatmeansthehashindexdoesnotsupportrangequeries.
AuniquehashindexhasanamortizedcomplexityofO(1)forlookup,insert,update,andremoveoperations.Thenon-uniquehashindexissimilar,butamortizedlookupperformanceisO(n),withnbeingthenumberofindexentrieswiththesamelookupvalue.
SkiplistIndex
Askiplistisasortedindextypethatcanbeusedtofindrangesofdocuments.
AnonymousGraphs
Youmayuseedgecollectionswithvertexcollectionswithoutthegraphmanagementfacilities.However,graphconsistencyisnotenforcedbythese.Ifyouremovevertices,youhavetoensurebyyourselvesedgespointingtothisvertexareremoved.Anonymousgraphsmaynotbebrowsedusinggraphviewerinthewebinterface.Thismaybefasterinsomescenarios.
Glossary
678