View
13
Download
0
Category
Preview:
Citation preview
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage2of27
DocumentRevisions
Date Change Insec/ons
January2019 IniCalpublicaCon -
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage3of27
ContentsDocumentRevisions
Introduc<on
Architecture
Prerequisites
SpecializedKnowledge
DesignConsidera<ons
SampleDataset
IAMRolesandPolicies
DeploymentOp<ons
TechnicalRequirements
Duringdeployment
Postdeployment
SystemRequirements
II.PlanningGuidance
Security
Costs
Sizing
III.DeploymentSteps
Step1.PrepareYourAmazonAccount
Step2.LaunchusingCloudForma<on
Step3.TesttheDeployment
Op<onal:UsingYourOwnDataset
Troubleshoo<ng
IV.Opera@onalGuidance
HealthCheck
BackupandRecovery
Rou<neMaintenance
EmergencyMaintenance
Support
SupportCosts
Appendix
Namingconven<on
RoleandPolicyDetails
2
4
5
6
6
6
6
7
7
7
7
8
9
9
9
9
10
11
11
11
16
19
20
21
21
22
23
23
23
23
24
24
24
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage4of27
I.IntroductoryMaterial
Features
Usecases
DocumentRevisionsAccelerite presents ShareInsights, an end-to-end big data analy?cs pla@orm that unifies differentanaly?csopera?onslikedataprocessing,storage,andvisualiza?oninonetool.
This deployment guide provides step-by-step instruc?ons for deploying Accelerite ShareInsights in theAmazoncloud.YoucanusethisShareInsightsenvironmenttoprocessandanalyzeyourowndatasetsanddashboards. This deployment provides prepopulated sample Wikipedia open data and sampledashboardstoanalyzeusingAmazonAthenaandAmazonGlueservices.
ShareInsightsmakesiteasyforanyonetoexplore,transformandvisualizebigcomplexdataonAmazonorHadoopdatalakesinminutes.
ShareInsightsonAmazonisdesignedtoleveragethepowerofAmazonAnaly?csservicessuchasAthena,andGluewithasimpledrag-and-dropvisualdesignerwhichenablesdataanalystswithnoprogrammingexper?setoeasilybuildtheirownadvanceddatapipelinesthatincludepowerfulvisualiza?ons.
ShareInsightsusesAmazonS3asadatalakeandusesthepowerofAmazonGlueCatalog.
• Easytousedrag-and-dropvisualpipelinebuilder• Visualdatalakeexplora?onwithslicinganddicing• Integrateddatavisualiza?onenginefordrag-and-dropdesigningofdashboards• FullyIntegratedwithAmazonAthena,AmazonGlue,andAmazonS3.• SupportsAmazonServerlessanaly?cstechnologiesforincrediblecostsavings• Integrateseasilywithmostpopularanaly?csrepor?ngtools(suchasTableauandQlik)• Easy-to-usedata-as-a-servicecapabili?es(usingRESTfulAPIs)
• Projectmanagersinaconsul?ngfirmslookingtoimproveu?liza?on,meetdeadlines,mi?gaterisksproac?velytoimprovecustomersa?sfac?onandcapacityu?liza?on.
• EnterpriseIT&dataengineerslookingtoautomatemigra?onofon-premiseETL,ELTjobstoAWSdataservicesrapidly.
• Datascien?stslookingtopreparedata,trainanddeploymodelsreliablyonAWS.• ManageddatalakeserviceprovidersonAWSlookingtoenabletheirbusinessuserswiththe
abilitytoderiveinsightswithano-codeanaly?cspla@orm.• TVchannelsandadver?serscanprocessreal-?meviewershipdatafrommul?plesourcesand
createcustomizeddashboardstounderstandandtargettheiraudiencebeYer.• Marke?ngmanagerslookingtoenhancecustomersegmenta?onandtargetcustomersfor
higherconversionandruneffec?vecampaignswithuniqueinsightsfromallrelevantdatasources.
• Clinicalresearcherslookingtoimprovedrugtrialenrolmenteffec?venessbyanalyzingpa?entdatawithaself-serviceinterface.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage5of27
Architecture
This ShareInsights deployment provides no-code analy?cs for data lakes that seamlessly u?lizes na?veAmazonanaly?csservicessuchasAmazonAthenaandAmazonGlue.Thisdeploymentbuilds followingenvironment in the Amazon Cloud including ShareInsights server, Amazon Athena, Amazon Glue andotherAmazonservices
Figure1:ShareInsightsarchitectureontheAmazonCloud
Thearchitectureincludesthefollowingservicesandcomponents:
•Amazon S3. Amazon Simple Storage Service (Amazon S3) is an object store that provides ar?factsnecessaryforShareInsights,includingdatasetstoconfigureAmazonAthenaandAmazonGluedatabaseand to compute aggregates for the sample dataset. ShareInsights creates a S3 bucket to keep itsintermediateandfinaldatasets.
•AmazonAthena.AmazonAthenaisaninterac?vequeryservicethatmakesiteasytoanalyzedatainAmazon S3 using standard SQL. The deployment creates Athena tables for each sample datasets.ShareInsightsallowsuser to select theanaly?csengine–AmazonAthenaandAmazonGlue - toeasilybuildtheirownadvanceddatapipelinesthatincludepowerfulvisualiza?ons.
•Amazon Glue. The Amazon Glue is a fully managed ETL (extract, transform, and load) service thatmakes it simple and cost-effec?ve to categorize your data, clean it, enrich it, and move it reliablybetweenvariousdatastores.AmazonGlueconsistsofacentraldata repositoryknownas theAmazonGlue Data Catalog, an ETL engine, and a job scheduler. ShareInsights creates a database to keep itsmetadata.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage6of27
•ShareInsightsonAmazonEC2.Thedeploymentprovidesasingle-instancedeploymentofShareInsightswith theability forDataanalysisandvisualizaBonusingAmazonAnalyBcsservicessuchasAthenaandGlue. It also provides Trial userweb page for geDng started, sample dashboards, training videos andassignmentsonShareInsights.
•AmazonVPC. The Amazon Virtual Private Cloud (Amazon VPC) service lets you provision a logicallyisolated secBon of the Amazon Cloud where you can launch resources in a virtual network that youdefine. The VPC provides a network architecture with mulBple public and private subnets that spanmulBpleAvailabilityZones,sothatAmazonresourcescanbedeployedinhighlyavailableconfiguraBons.ThisdeploymentusesasingleAvailabilityZone.
•IAMroles.ThisdeploymentconfiguresAmazonIdenBtyandAccessManagement(IAM)rolestoprovidetherequiredaccessforAmazonresourcescreatedduringdeploymenttoaccessotherAmazonresourceswhenrequired.TheseIAMrolesenableaccesstodatainAmazonS3,enableAmazonAthenaandAmazonGluetoanalyzedataandsaveresultsonAmazonS3.
•Sampledataset.ThisShareInsightsdeploymentprovidesasampledataset,availableinAmazonS3,andcreatesAthenatablesundernewlycreatedGluedatabaseforShareInsights.
Prerequisites
Beforeyoubegin,reviewtheEndUserLicenseAgreement(EULA).
DesignConsidera@ons
SpecializedKnowledge
BeforeyoudeployAcceleriteShareInsights,werecommendthatyoubecomefamiliarwiththefollowingAmazonservices.(IfyouarenewtoAmazon,seeGe#ngStartedwithAmazon.)
SampleDataset
This ShareInsights deployment includes a dataset from aWikipedia open data. The dataset includesWikipediaopendata forWhatHappens inMonth,PageView,andMoviesBoxOfficeanalysis.TablesrelatedtothesedatasetsarecreatedinAmazonGluedatabase.Thisdeploymentallowsusertoanalyzethesesampledatasetsandviewsampledashboards.
•AmazonS3•AmazonEC2•AmazonAthena•AmazonGlue•AmazonVPC•AmazonIAM
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage7of27
IAMRolesandPolicies
Thisdeploymentcontainsnon-sensi>ve,sampledatasetthatdoesn’tneedtobesecured.However,youcanusealterna>vedatasetswithprivate, sensi>vedata.Thedeploymentcreates the following threeIAMrolestoensurethatprivatedataismanagedsecurely:
DeploymentOp8ons
ShareInsightsprovidesonlyonedeploymentop>onusingAmazonCloudForma>on.
Duringdeployment
AmazonCloudForma>oncreatesfollowingresourcesrequiredforShareInsightsenvironment:
• EC2AssumeRole. This IAM roleprovidesAmazonS3access to the referenceddataset fromEC2instances launchedwithin the deployment. It also enables launched EC2 instances to associate anIAMrolewithrequiredAmazonservicessuchasAmazonAthena,AmazonGlue.BecausethisroleisintendedtobeassociatedwithEC2 instances, thedeploymentalsocreatesan IAM instanceprofilethatincludesthisIAMrole.
• GlueServiceRole.ThisIAMroleprovidesAmazonGlueaccesstoAmazonEC2instance.
• LambdaExecu8onRole.This IAMroleprovidesAmazonLambdaaccess toAmazonEC2 instance,AmazonS3andotherAmazonservices.
• EC2InstancePolicy.Thispolicy isforAmazonAssumeroletobeusedbyAmazonEC2toconnectwithotherAmazonservices.
• IAMPassRolePolicy.ThisIAMpolicyallowAmazonEC2instancetopassaroletootherAmazonservices.
ReferRoleandPolicyDetailsunderAppendixforrole,policyandpermissiondetails.
TechnicalRequirements
This ShareInsights deployment includes one Amazon CloudForma>on templatewhich creates a virtualprivatecloud(VPC)withapublicsubnet,whichisusedtohousetheremainderofthecomponentsinthearchitecture.
1. AmazonVPC2. AmazonPublicsubnet3. AmazonVPCendpointforAmazonS34. AmazonInternetGateway5. AmazonPublicRouteTable6. AmazonSecurityGroup7. AmazonIAM
• ReferIAM Roles and Policiesformoredetails.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage8of27
8.AmazonS3•OneAmazonS3privatebucketforShareInsightsdatalake•SharedopendatafromAmazonS3publicbucket (hEps://s3.amazonaws.com/shareinsights-pub/)will be used to create Athena tables undernewlycreatedShareInsightsAmazonGluedatabase.Thisdataisusedforsampledashboards.
9.AmazonGlue•OneShareInsightsdatabasewillbecreated•CreateAmazonGlueconnecMon
10.AmazonEC2instance•Tocreateinstance•PermissiontoaEachAmazonEC2instanceprofileassumerole
11.AmazonLambdafuncMonsfor:•CreaMnguniqueresourcename:
i.IncludingwithMmestamptoallowusertocreatemulMpleinstancesinoneaccountii. NormalizenameforspecialcharactersandlengthforAmazonS3,AmazonIAMroles,AmazonGlueconnecMon
•ShareInsightsspecificAmazonS3bucketcleanup•GeXngmainAmazonRouteTableid
ReferappendixforresourceNamingconven,on.
1.AmazonEC2•Edge/GatewaynodehostsShareInsightswebserver• ShareInsights serves user request using Amazon services such as Athena, Glue, S3 andCloudWatch.•DuringrequestAmazonAssumeRoleisused
2.AmazonAthena•ShareInsightswillcreate/delete/modifyAmazonAthenatablesandviewsduringexecuMon
3.AmazonGlue•BasedondashboardsandanalyMcstransformaMonused:
1.Create,deleteandmodifyAmazonGluetables2.Create,deleteandmodifyAmazonGlueJobs
4.AmazonS3•ShareInsightsusesAmazonS3asDataLake•DuringexecuMonShareInsightsreadsandwritesAmazonS3datausingAmazonAthenatables
orviews• DuringexecuMonofdashboardtheintermediateandfinaldatasetsaresavedasAmazonS3 objects. ShareInsightsalsocreatesappropriateAmazonAthena tablesandviewson topofthesefiles.TheseAthenaobjectsarepartofShareInsightsGluedatabase.
5.CloudWatch•JobexecuMonlogginginformaMoniscollectedinAmazonCloudWatch
Postdeployment
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage9of27
SystemRequirements
Amazon private key file with appropriate permissions to Amazon resources men@oned in Role andPolicyDetailssec@onunderappendix.
II.PlanningGuidance
Security
Rolesandpoliciesarecreatedforsecuritypurpose.ReferIAMRolesandPoliciesforthelist.
Forpermissiondetails,referRoleandPolicyDetailsunderAppendix.
ShareInsights user login andpassword is authen@cated internally. Thepassword is stored in encryptedformatwithinShareInsightsdatabase.
AbovecostdoesNOTincludeusagecostofAmazonservicesusedbyShareInsights.
YouareresponsibleforthecostoftheAmazonservicesusedwhilerunningthisdeployment.TheAmazonCloudForma@on templates include configura@on parameters that you can customize. Some of theseseKngs,suchasinstancetype,willaffectthecostofdeployment.SeethepricingpagesforeachAmazonserviceyouwillbeusingforcostes@mates.
TheAcceleriteShareInsightsdeploymentallowsyoutobuildadvancedbigdatapipelinesanddashboardsusingAmazonAthena andAmazonGlue engine.However, you are responsible for the cost ofAmazonresources used by ShareInsights. Following is basic informa@on about the Amazon services used byShareInsights:
Costs
FollowingistheShareInsightscostbasedonEC2instancetype:
Instancetype So=ware/Hour(USD$)
m4.xlarge
r4.xlarge
m4.2xlarge
r4.2xlarge
m4.4xlarge
r4.2xlarge
$0.20
$0.27
$0.40
$0.53
$0.80
$1.06
$4.20
$4.27
$5.40
$5.53
$6.80
$7.06
$4
$5
$6
ApproximateEC2/Hour(USD$)
TotalCost/Hour(USD$)
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage10of27
•AmazonAthenaoYouarechargedbasedontheamountofdatascannedbyeachqueryroundeduptothenearest megabyte, with a 10MB minimum per query. Amazon Athena charges USD$5 per TB of datascanned.oFormoreinformaLonrefertoAmazonAthenapricinginforma1on
•AmazonGlueoYouarechargedbasedontheLmeyourETLjobtakestorunandthenumberofDataProcessingUnits(orDPUs)usedtorunyourETLjob.oAmazonGluecatalogchargeswillbebasedonnumberofobjects.AnobjectintheAmazonGlueDataCatalogisatable,tableversion,parLLon,ordatabase.oFormoreinformaLonrefertoAmazonGluepricinginforma1on
•AmazonS3oYouwillbechargedbasedonamountofdataandlocaLonofS3bucketoFormoreinformaLonrefertoAmazonS3pricinginforma1on
•AmazonCloudForma8on(duringdeploymentonly)oAsofnowthereisnoaddiLonalchargeforAmazonCloudFormaLonoFormoreinformaLonrefertoAmazonCloudForma1onpricinginforma1on
•AmazonCloudWatchoShareInsightsusesCloudWatchonlyforlogscollecLon.TheapproximatelogsrelatedcostisUSD$0.535/GB.oFormoreinformaLonrefertoAmazonCloudWatchpricinginforma1on
•AmazonLambda(duringdeploymentonly)o ShareInsights creates five Lambda funcLon and uses them only during stack creaLon. Each LambdafuncLoniscalledonlyonceduringcreaLon.OnlyoneLambdafuncLoniscalledduringstackterminaLon.o ApproximatecostwillbelessthanUSD$1o FormoreinformaLonrefertoAmazonLambdapricinginforma1on
Sizing
Usethefollowingguidelinestoselectyourinstancesize:
Instancetype
m4.xlarge/r4.xlarge
m4.2xlarge(recommended)/r4.2xlarge/c5.2xlarge/t2.2xlarge
m4.4xlarge/r4.4xlarge/c5.4xlarge
4GB-8GB
8GB-16GB
16GB-64GB
Averagedatasizeworkload
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage11of27
III.DeploymentStepsThisguidestepsthroughtherequirementsandprocessfordeployingShareInsightsthroughtheAmazonMarketplace
Step1.PrepareYourAmazonAccount
1.Ifyoudon’talreadyhaveanAmazonaccount,createoneathEps://aws.amazon.combyfollowingtheon-screeninstrucKons.
2.UsetheregionselectorinthenavigaKonbartochoosetheAmazonRegionwhereyouwanttodeployShareInsightsonAmazon.Refertosupportedregions.
3.Createakeypairinyourpreferredregion.
4. Ifnecessary,requestaservicelimit increase fortheAmazonEC2m4.xlargeinstancetype.Youmightneed todo this if youalreadyhaveanexisKngdeployment thatuses this instance type,andyouthinkyoumightexceedthedefaultlimitwiththisreferencedeployment.
5. Ifnecessary,requestaservicelimitincreaseforAmazonCloudFormaKonstacks.Thisdeploymentwillcreateupto1newstackwithinyouraccount.
Step2.LaunchusingCloudFormaAon
NoteYouareresponsibleforthecostoftheAmazonservicesusedwhilerunningthisdeployment.
1. Launch theAmazonCloudFormaKon template into yourAmazon account. SeeDeploymentOp7onsearlierinthisguide.
Deploymenttakesabout25minutestocomplete.
2. Check the region that’s displayed in the upper-right corner of the navigaKon bar and change it ifnecessary.CurrentlyShareInsightssupportsthreeregions:
a.USEast(N.Virginia)b.USEast(Ohio)c.USWest(Oregon)
3.OntheSelectTemplatepage,keepthedefaultse]ngforthetemplateURL,andthenchooseNext.
4.OntheSpecifyDetailspage,reviewtheparametersforthetemplate.Entervaluesfortheparametersthatrequireyourinput.Forallotherparameters,youcancustomizethedefaultse]ngsprovidedbythetemplate.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage12of27
StackConfigura1on:
Parameterlabel(name)
Stackname Requiresinput
Default
AmazonInfrastructureConfigura1on:
Parameterlabel(name)
Keypairname(KeyName)
Requiresinput
Default
Specifyastackname.
Instancetype(InstanceType) m4.xlarge EC2instancetypefortheShareInsightsserver.
Descrip1on
Public/privatekeypair,whichallowsyoutoconnectsecurelytoyourinstanceaLeritlaunches.WhenyoucreatedanAmazonaccount,thisisthekeypairyoucreatedinyourpreferredregion.
Descrip1on
ShareInsightsConfigura1ons:
Parameterlabel(name)
AdminPassword(Password) Requiresinput
Requiresinput
Default
ConfirmAdminPassword(PasswordConfirm)
ConfirmadministratorpasswordforShareInsights.
ThepasswordoftheadministratorforShareInsights.NOTE:Theusernameoftheadministratoris“shareinsights”.
Descrip1on
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage13of27
5. ChooseNext. 6. OntheOp,onspage,youcanspecifytags(key-valuepairs)forresourcesinyourstackandset
advanced opFons.Whenyou’redone,chooseNext.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage14of27
7. OntheReviewpage,reviewandconfirmthetemplateseCngs.UnderCapabili/es,selectthecheckboxtoacknowledgethatthetemplatewillcreateIAMresources.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage15of27
Figure2:CheckboxforIAMresources
8. ChooseCreatetodeploythestack.
9. Monitorthestatusofthestack.WhenthestatusisCREATE_COMPLETE,thedeploymentiscomplete.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage16of27
Step3.TesttheDeployment
1.LandingpageURL
2.ShareInsightsdocumentaBonURL
3.EC2details
a.SSHinformaBontoconnecttoEC2instance.
4.Trialuser(forgeIngstartedonShareInsights)
a.Logindetails•TrialuserIDandpasswordwithreadonlyaccess
b.URLdetails•ItincludesbasicinformaBonaboutShareInsightsandfewexercisestolearnShareInsights
AOertheinstallaBoniscomplete,theOutputsecBoninCloudFormaBonwillsharethefollowinginformaBon:
Figure3:Outputsec=oninCloudForma=onaAerinstalla=oniscomplete
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage17of27
Totestthedeployment:
1. NavigatetotheShareInsightsServerbypoinCngawebbrowsertotheURLforLandingPageUrlintheOutputstab.
2. ItshoulddisplaythreedashboardsunderDashboardExplorerforthesampleWikipediadata.
3. ViewtheDataLakeExplorerwithinShareInsightsServer.TheCloudFormaContemplateconfiguressampledatasetswhichwillbelistedunderPublishedDatasetssecCon.
Figure4:ShareInsightsLoginpage
Figure5:ShareInsightsHomepagedisplayingDashboardExplorer
4. ViewthedashboardsthathavebeenpublishedtoShareInsightsServerfromthesampledataset.ThedashboardsaredesignedtocombineAmazondatasourcesandusethearchitectureinpowerfulwayswhileretaininggoodperformanceasthevolumeofdata,numberofusers,andscaleofqueriesincreaseoverCme.
5. VisittheTrialUserURLforgeTngstartedonShareInsights,trysampledashboards,viewingtrainingvideosandtryingassignments:
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage18of27
Figure6:ShareInsightsTrialUserPage
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage19of27
Op'onal:UsingYourOwnDataset
FollowingarethestepstoingestdatainShareInsightsusingAthenaandGluetables:
1. Createanewdashboard.
2. FromthePublishConfigura'onopCons,select"ExecuConEngine"as"athena"or"gluespark".
3. SwitchtoAdvancedEditmode.Youcandoitbyremovingthetrailing"f"fromtheword"ediL"inthesampleURLbelow.h"ps://<hostname>/live/v1/dashboards/my_dashboard/edi8
4. AddaDataSec'on(i.e.DsecCon)toaddthedetailsofthesourcedata.Hereisanexamplewithquerybinding:
D: mytable_ds_1 : [language_code(number),greeting] # Add table columns mytable_ds_2 : [currentvalue(number),brand,sector] # Add table columns
D.mytable_ds_1 : # dataset name to be used protocol : hivequery source : select * from my_glue_db.mytable1 # mytable : Name of the Table. limit_rows : 1000 # This is optional publish : 'myNewDataset1' # Add this flag to publish the dataset in ShareInsights lake and to be consumed by other dashboards
D.mytable_ds_2 : # dataset name to be used protocol : hivequery source : select * from my_glue_db.mytable2 # mytable : Name of the Table. limit_rows : 1000 # This is optional publish : 'myNewDataset2' # Add this flag to publish the dataset in ShareInsights lake and to be consumed by other dashboards.
5. ToSavethedashboardclickthesaveicononupper-rightcornerorpressCtrl+Sonthekeyboard.
6. ToPublishthedashboardclicktheexecuteicononupper-rightcorner,orpressAlt+Ronthekeyboard.Youcanviewthelogsasthedashboardisbeingpublished.OncetheexecuConisoverthelastlinewillappearas:“Reloadingdataexplorer”.
ThedatasetswithpublishfieldswillbeavailableintheShareInsightsDataLakeExplorerforconsumpConinotherdashboardsforallShareInsightsusers.
TolearnmoreaboutLanguageReference,refertoShareInsightsHelponupper-rightcorner.
Figure7:HelpmenuunderUsernamemenu
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage20of27
Troubleshoo-ng
Q.IencounteredaROLLBACK_COMPLETEstatusaEerIlaunchedthedeployment.WhatshouldIdo?
A.ThisoccurswhenAmazonCloudFormaKonisnotabletocreateanAmazonresourcewhichisrequiredfordeployingShareInsights.FollowingarethestepstofindtheresourcewhichfailedduringcreaKon:
1.VisitAmazonCloudFormaKonConsole2.Selectthestackyouweretryingtocreate3.SelectEventstab4.SearchforeventwithStatusasCREATE_FAILED
5. LookforthereasonmenKonedintheStatusreasoncolumn Commonreasonsforfailureare:
1.InsufficientIAMPermissions(notauthorizedtoperform)•YoushouldhavepermissiontocreateresourcesrequiredforCloudFormaKontosuccessfullycreateShareInsightsenvironment.ForlistofresourcesreferDuringdeploymentsecKon.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage21of27
2. Exceededlimit
• Increasethelimitofiden?fiedresourcetypefromStatusreasoncolumn(men?onedabove)and relaunchtheAmazonCloudForma?ontemplate.
3. ResourceFailedtoStabilizeDuringaCreate,Update,orDeleteStackOpera>on
• Probablereasonforthisfailuretooccuristheresourcedoesnothavesufficientpermissionsand thecrea?onhasfailed.
Foraddi?onalinforma?on,seeTroubleshoo>ngAmazonCloudForma>onontheAmazonwebsite.
Q. AmazonCloudForma?onissuccessfulbuttheShareInsightsURLisnotresponding.WhatshouldIdo?
A. ReferEmergencyMaintenanceforinves?gate.
IV. Opera/onalGuidance
HealthCheck
AOerinstallingandloggingintoShareInsights,clickonthe“DataLakeExplorer”tabwhichshouldlistafew
datasetsunderthe“PublishedDatasets”sec?on.Youshouldseesomeofthedatasetsshownbelow.
Figure8:ShareInsightsDataLakeExplorertabwithPublishedDatasetssec/ononleJ
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage22of27
Alterna<vely,youcanautomatethisbyusingAmazonRoute53.FollowingarethestepstocreateHealthCheckusingAmazonRoute53:
1.VisitAmazonRoute53console
2.ChooseCreatehealthcheck
3.GivehealthcheckaName
4.SelectDomainnameunderSpecifyendpointby
5.SelectHTTPSProtocol
6.EntertheAmazonEC2PublicDNS(ShareInsightsserver)inDomainnamefield
7.SetPathfieldto"live/login"
8.UseAdvancedconfiguraAonseXngstocustomize,ifany
9.ChooseNext
10.Togetno<fiedonfailure,selectYestoCreatealarm.SelectNo,ifyouwishtocheckhealthonAmazonRoute53console.
11. ChooseCreatehealthcheck
12. YoucanseeShareInsightsserverhealthonAmazonRoute53consolea]erfewminutesofcrea<onundertheName(instep#3).
IncaseifShareInsightsisnotresponding,followstepsmen<onedinEmergencyMaintenancesec<on.
NoteYouareresponsibleforthecostoftheAmazonRoute53services.ReferAmazonRoute53pricing.
BackupandRecovery
ShareInsightsteamrecommendaweeklybackupofEBSvolumes.PleaseseetheAmazonrecommenda<onforbackupandrecoveryusingasnapshotoftheEBSvolumes.
Recovery:
1. Torecoveranydashboardsearchfordashboardnamefolderunder/data/context/shareinsights/dashboards/inthebackups.
2. Forcompleterecoverycopy/data/context/shareinsights/folderfromthebackup.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage23of27
Rou'neMaintenance
ShareInsightshasnorou<nemaintenance.ShareInsightsdoesnothaveanykeysandthereforedoes
notrequireanykeyrota<onasitworksontheAmazonAssumeRolepolicy.
AnewversionofShareInsightsAMIwillbeavailableeveryquarter.Upgradeinstruc<onswillbeshared
witheveryrelease.TheAmazonmarketplacewillupdateyouifthereisanupgradeavailable.Contact
ShareInsightssupportforhelp.
EmergencyMaintenance
ShareInsightsisavailableinlimitedAmazonregionsasitrequiresAmazonAthenaandGlueservices.
Refertosupportedregions.
IftheShareInsightsURLisnotresponding,thentrythefollowing:
1. CheckonAmazonEC2consoleiftheEC2instanceisupandrunning.Ifnot,thenstartitand
itwillrestarttheShareInsights.
2. IncaseifyouarenotabletostartEC2instanceoritisnotaccessiblethenpleasecheckif
Amazonservicesareupforyourregionandavailabilityzone.
3. IncaseifEC2isupandrunningbutShareInsightsisnotthenrestarttheEC2machine.This
willstartShareInsightsinapproximately2-3mins.AlternatelysshEC2machineandexecute
sudosystemctlrestartlivecommand.
4. IfnoneoftheabovestepsworkthensendallthefilesfromfollowingAmazonEC2instance
folderstoShareInsightssupport(referSupport):
• /data/contexts/shareinsights/logs/<<host-name>>/• /data/contexts/shareinsights/sta<c/context/live/
ThereshouldnotbeanylossofShareInsightsdashboardordata(whicharesavedonAmazonservices
suchasS3,AthenaandGlue).IncaseofanylossofShareInsightsdashboardpleaserecoverfromthe
lastbackuptakenasexplainedinBackupandRecoverysec<on.
Support
Pleasecontactusforsupport(shareinsights_support@accelerite.com).withyourqueryandcontact
details.
SupportCostsSupportcostswilldependonthecontract.
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage24of27
Appendix
Namingconven3on
Resource names are created using the “shareinsights-<<stack name>>-<<Bmestamp>>” template. This
ensuresproperfuncBoningofmulBpleShareInsightsinstancesinthesameAmazonregion.
Forexample,ifusergivenstacknameis“shareinsights-stack-1”thenresourcenameswillbe:
RoleandPolicyDetails
Rolescreated:
1. ShareInsightsEC2InstanceRole
2. ShareInsightsGlueServiceRole
3. ShareInsightsLambdaExecuBonRole(usedduringdeploymentonly)
Policiescreated:
1. ShareInsightsEC2InstancePolicy
2. ShareInsightsIAMPassRolePolicy
3. ShareInsightsGlueServicePolicy
RoletoPolicyMatrix:
First35charactersofstacknameisconsideredforcreaBngtheresourcenameduetoAmazonresource
namelimitaBons.
Resource
S3bucketname
Name
shareinsights-shareinsightsstack1-1548420158
AccessRole shareinsights_shareinsightsstack1_1548420158_AR
GlueServiceRole shareinsights_shareinsightsstack1_1548420158_GSR
Gluedatabasename shareinsights_shareinsightsstack1_1548420158
GlueconnecBon shareinsights_shareinsightsstack1_1548420158_redis_conn
Policyname
ShareInsightsEC2InstancePolicy
ShareInsightsEC2InstanceRole ShareInsightsGlueServiceRole
ShareInsightsIAMPassRolePolicy
ShareInsightsGlueServicePolicy
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage25of27
ShareInsightsEC2InstancePolicydetails:
Service
AmazonAthena
AmazonS3 arn:aws:s3:::<source_data_bucket>/*,arn:aws:s3:::<si_bucket_name>/*
StartQueryExecuJon
GetNamespace
GetQueryResults
GetCatalogs
GetNamespaces
CreateNamedQuery
GetExecuJonEngine
StopQueryExecuJon
PutObject
GetObject
ListBucket
DeleteObject
GetBucketLocaJon
GetLogEvents
Ac;on Resource
ShareInsightsIAMPassRolePolicydetails:
Service
GetRole
PassRole
Ac;on Resource
*
GetDatabase
GetDatabases
GetJobRun
GetTable
GetTables
CreateJob
CreateTable
DeleteJob
StartJobRun
*
AmazonCloudWatchLogs
AmazonGlue
IAM
arn:aws:logs:*:*:log-group:*
arn:aws:iam::<<account_id>>:role/ShareInsightsGlueServiceRole
ShareInsightsontheAmazonCloud
AcceleriteShareInsightsDeploymentGuidePage26of27
ShareInsightsGlueServicePolicydetails:
Service
CloudWatch PutMetricData
AssociateKmsKey
CreateLogStream
PutLogEvents
arn:aws:logs:::/aws-glue/*
arn:aws:logs:::/aws-glue/*
arn:aws:logs:::/aws-glue/*
Ac;on Resource
*
DescribeNetworkInterfaces
DescribeRouteTables
DescribeSecurityGroups
DescribeSubnets
DescribeVpcAMribute
DescribeVpcEndpoints
CreateNetworkInterface
ListAllMyBuckets
ListBucket
GetBucketAcl
GetBucketLocaOon
GetObject
DeleteObject
PutObject
DeleteNetworkInterface
DeleteTags
CreateTags
*
*
CloudWatchLogs
EC2
S3
"arn:aws:ec2:*:*:network-interface/*","arn:aws:ec2:*:*:security-group/*","arn:aws:ec2:*:*:instance/*"
"arn:aws:s3:::<bucket_name>""arn:aws:s3:::<bucket_name>/*"
GetConnecOon
GetConnecOons
GetDatabase
GetDatabases
GetParOOon
GetParOOons
GetTable
GetTables
*
Glue
Recommended