Upload
others
View
13
Download
0
Embed Size (px)
Citation preview
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 1
D4.3Roadmaptoacloudmarketstructureencouragingtransparentcloudpricing–
Finaliteration
www.CloudWatchHUB.eu
ThefocusofthisdocumentistohighlighthowbesttomanagesystemicrisksintheInfrastructure-as-a-Service market that are noteworthy to stakeholders seeking to shape policies for a fair andtransparentcloudmarket.Thisdocumentoutlineshowthecloudcomputingmarket is structured,why it is essential for Europeand the typesof activitiesnecessary to createa fair, balanced, andtransparent market. This document makes recommendations to key stakeholders for actions tomitigaterisksofmarketfailure,anddeliverastablecloudcomputingmarket.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 2
CloudWATCH2Mission
It is only when the innovation process is inclusive and open that we truly advance technology forhumanity – from small businesses to public sector organisations and citizens as the new digitalconsumers.Theuseofopensourcesoftwareandopenstandardsarebecoming increasinglyseenasenablersand levellers forpublicandprivate sectorsalike,bundling skills to createnewservicesandapplications.
Tosupport thisCloudWATCH2takesapragmaticapproachtomarketuptakeandtheexploitationofresults coming from European sustainable competitiveness for wider uptake and commercialexploitation.ItprovidesasetofservicestohelpEuropeanR&IinitiativescapturethevaluepropositionandbusinesscaseaskeytoboostingtheEuropeaneconomy.
CloudWATCH2servicesinclude:
● A cloudmarket structure roadmapwith transparent pricing to enable R&I projects to chartexploitationpathsinwaystheyhadnotpreviouslyconsidered,orhelpthemavoidapproachesthatwouldnothavebeensuccessful.
● Mapping theEUcloudecosystemofproducts, services and solutionsemerging fromEUR&Iprojects.Identifyingsoftwarechampionsandbestpracticesinmitigatingrisksassociatedwithopensourceprojects,andultimately,enablefastertime-to-valueandcommercialisation.
● Impactmeetingsforclusteringandconvergenceoncommonthemesandchallenges.Re-useoftechnologieswillalsobeofparamountimportance.
● Promotingtrusted&secureservicesthroughroadshowsanddeepdivetrainingsessions.GivingR&IinitiativesaroutetousersatmajorconferencesorinlocalICTclusters.
● AportfolioofstandardsforinteroperabilityandsecuritythatcanfacilitatetherealisationofanecosystemofinteroperableservicesforEurope.
● Cloud interoperability testing in an international developer-oriented and hands-onenvironment.Findingswillbetransferredintoguidancedocumentsandstandards.
● Riskmanagementandlegalguidelineswithpracticalexamplesofcloudcontracts’clausesthatneed to be assessed before purchasing cloud services to the cloud for private and publicorganisations to lower barriers and ensure a trusted European cloud market.
Disclaimer
The CloudWATCH2 (Think Cloud Services for Government, Business and Research) is funded by theEuropean Commission’s Unit on Software and Services, Cloud Computingwithin DG Connect underHorizon2020.
Theinformation,viewsandtipssetoutinthispublicationarethoseoftheCloudWATCH2Consortiumand itspoolof internationalexpertsandcannotbeconsidered to reflect theviewsof theEuropeanCommission.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 3
Document Information Summary
Documenttitle: D4.3Roadmaptoacloudmarketstructureencouragingtransparentcloudpricing–finaliteration
MainAuthor(s): JohnWoodley&JamesMitchell,StrategicBlue
Contributingauthor(s): FrankKhanSullivan,StrategicBlue
Reviewer(s): Nicholas Ferguson & Silvana Muscella, Trust-IT; David Wallom &MichelDrescher,UOXF
Targetaudiences: Cloudproviders,cloudresellers,majorcloudbuyers,aswellasthosewhomayhaveregulatoryjurisdictionovercloudmarkets
Keywords: Cloud Computing, Infrastructure as a Service, Market, Roadmap,CloudPricing,Regulation,EuropeanCloudInitiative,DigitalAgenda,DigitalSingleMarket
Deliverablenature: Report
Disseminationlevel:(Confidentiality)
PUBLIC
Contractualdeliverydate: M23(July2017)
Actualdeliverydate: 28July2017
Version: V1(FinalVersion)
Referencetorelatedpublications
https://ec.europa.eu/digital-agenda/en/european-cloud-initiativehttps://ec.europa.eu/digital-agenda/en/european-cloud-initiative
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 4
ForewordTheprovisionofITservicesviathecloudisbecomingthenewnorm.Enterprisessmallandlarge,aswellasmanypublic sector organisationsaremoving towardsa “cloud first policy”, craving theagility that on-demand ITallows. TheEuropeanCommissionhas identified the strategic roleof cloudwithin itsDigitalSingleMarketStrategyforEuropeandhasset-upacoherentstrategyinordertoacceleratethetake-upandincreaseduseofcloudcomputingacrossalleconomicsectors.
Europewantstoembraceallthebenefitsofferedbycloudtechnologies.Forthispurpose,interoperabilitymustbeenabledandrelevantstandardsmustbeleveraged.Thisisneededtoallowbenchmarkingofservicequalityandpricecomparison.Emergingissuesrelatedtoownership,access,portingofdataandswitchingofcloudserviceprovidersshouldbeadequatelyaddressed.Inthisway,wecanensurealevelplayingfieldforallcloudplayerswhichwillstimulatecompetitionandcreateinnovativeandefficientmarketplacesforusersofcloudservicesinEurope.
PierreChastanet,ActingHeadofUnit,Cloud&Software,EuropeanCommission.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 5
Table of Contents
Document Information Summary..........................................................................................3
Foreword....................................................................................................................................4
1 Introduction.......................................................................................................................7
2 WhyMarketStructureMatters.........................................................................................9
3 Howwegothere.............................................................................................................13
4 TheRoadinfront.............................................................................................................15
5 MakingthecaseforCloudTrading..................................................................................16
6 EuropeanInitiativespavingtheway................................................................................18
6.1 DigitalSingleMarket.............................................................................................................18
6.2 EuropeanOpenScienceCloud..............................................................................................19
6.3 ProcurementInnovationforCloudServicesforEurope(PICSE)...........................................20
7 PotholestoAvoid............................................................................................................20
7.1 OptimalStandardisation........................................................................................................20
7.2 Withgreatpowercomesgreatresponsibility.......................................................................21
7.3 Rebalancingtheasymmetryinmarketinfluence..................................................................22
8 RecommendationsfortheEuropeanCommission..........................................................23
9 MethodologyforResearchingthisRoadmap..................................................................24
10 Conclusion.......................................................................................................................25
LogTable..................................................................................................................................26
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 6
TableofFiguresFigure1BlackSwanevents............................................................................................................................7
Figure2Electricitysupplymarketsharesbycompany:Domestic(GB).......................................................11
Figure3Themarketstructureismaturing...................................................................................................12
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 7
1 IntroductionInApril2016,wepublishedaPreliminaryVersionofthisRoadmap,highlightinganumberofriskstotheglobalcloudcomputingmarketandhencetothedigitaleconomiesthatrelyuponitson-demandcomputingcapabilities. At that time, itwas very unclear as towhether themarket structurewould evolve into ahealthy,sophisticatedmarketthatencouragescompetition,orifitsevolutionwouldstallasaresultofthedominantprovidersprotectingtheirinterests.
AsDrVinceKellen,CIOofUniversityofCaliforniaatSanDiegopointedout:
“Thecloudmarketisgrowingrapidlyinarushofsometimesirrationalexuberance.Aswehaveseensomanytimesbeforeinotherindustries,thismaybejustapatchofsunshinebeforeaperfectstormofunmanageable risk rushes in.Whatweneed rightnowareexperiencedmindswith the rightimagination toanalyzewhat feware talkingabout: black swanevents in theburgeoning cloudmarket.“
Figure1BlackSwanevents
Morerecently,othercommentatorshaveindependentlyvoicedrelatedconcerns,somegoingsofarastocall for the breakup of some of the titans of the technology industry, includingGoogle, Facebook andAmazon.Suchanextremeuseofanti-trustlawswoulditselfbeaBlackSwanevent,asitcouldresultinsignificant disruption to the direct and indirect user base of the affected services. There are alsocommentatorswho argue strongly that competition is alive and kicking. As Tim Harford, a respectedeconomist,commentsontheissueintheFinancialTimes,“Thepolicyresponserequiredissubtle:afterall,the growth of innovative, productive companies is welcome. It’s the unintended consequences of thatgrowththatposeproblems.”
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 8
This matters because the European Commission (EC) has (rightly) identified digitisation of Europeanindustryasakeypriority,withover€50billionoffundingalreadysetasidetoencourageitthroughdigitalinnovationprojects,manyofwhichare likely to leveragetheservicesof theabovementionedtitans,orthosecompanieswiththedauntingtaskofcompetingagainstthem.
TheEC’sDigitalSingleMarket(DSM)inEuropeisanunprecedentedopportunitytocreateoneofthebiggestdigitalmarketplacesintheworldwhichcanhaveahugelypositiveimpactonEurope’scompetitivenessandproductivityacrossindustrialandservicesectors.TheEChasidentifiedfiveprioritydomainswhicharethebuildingblocksoftheDSM,namely:Cloudcomputing;5G;internetofthings(IoT);cybersecurity;anddata.With growing convergence of these technologies giving value to digital systems, transparency andappropriatestandardisationisurgentlyrequiredsothattheDSMisfoundedontrustworthysolutionsbasedon interoperable systems and interfaces that keep markets open, boost innovation and allow serviceportability.
WearepleasedtoreportthatmanyoftherisksweidentifiedinthePreliminaryVersionarereducing,andothersarebeingmitigatedwithsensiblestrategies.Themostpromisingofallisthematurityofthesupportofferedtotheecosystemofcloudintermediaries,i.e.cloudresellersandmanagedserviceproviders,thatprovidediverseandcompetitivewaysforcloudbuyerstoprocurecloudservicesbasedonthedominantcloudprovidersaswellasaggregatingthebuyingpowerofnumerous,diversecloudusersandusingthattoprotecttheirinterests.
ThestructureoftheInfrastructure-as-a-Servicebaselayerofthecloudmarketismaturingrapidly,andingeneralthisisleadingatleasttorationalpricing,ifnotfulltransparencyinpricing,asaresultofinherentdifficulties in making like-for-like technology migrations or even just comparisons. Again, specialistintermediatorsactingas“cloudsuppliers”havethemotivationandtheexpertisetoanalyseandcomparethemultitudeofwholesaleIaaSproductofferingsinsupportoftheirend-usercustomers.
Thereisnowamarketthathasevolved,withoutsignificantregulatoryintervention,tolooknotdissimilarto sophisticated, regulatedmarkets such as electricity. Here is what Germany’s energy regulator, theBundesnetzagentursaysaboutitsmarket:
“Well-functioningwholesalemarketsarefundamentaltocompetitionintheelectricitysector.Spotand futures markets are crucial for meeting suppliers' short and longer term electricityrequirements. Power exchanges play a key role alongside bilateral, over-the-counter (OTC)wholesaletrading.Theycreateareliabletradingforumandatthesametimeprovideimportantpricesignalsformarketparticipantsinotherelectricitysectors.”
Thereareanumberofdominantgenerationtechnologiesbeingusedtoprovidemostofthecloudcapacitythat is inuse,but therearestillothers inoperationatasmallerscale.Buyersdonothavetopurchasecapacitydirectlyfromthedominanttechnologyprovider,butratherhavealargechoiceofintermediariestopurchase from,oftenwithother technicalor financial value-added services included in theoffering.Theredoremainmorebarrierstoswitchingbetweentechnologies inthecloudmarketthanin,say,theelectricitymarkets,howeverswitching“supplier”,i.e.whoinvoicesthecustomer,isjustasstraightforward.Forthosechargedwiththechallengingroleofregulatoryoversightofthismarket,theauthorsrecommendthatthefocusshouldbeonencouraginghealthycompetitionatthepointofsupplytothecustomer,andensuringthatcloudintermediariesallhavethesamefairaccesstocompetitivecloudpricingasthedirectsalesor“supplier”partoftheincumbentcloudproviders.Thiswillleavethedooropenformarketentryby
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 9
challenger cloud providers in the future, once technologies and other initiatives that make switchingprovidereasierhavematuredfurther.
Thenextstageofcloudmarketevolution,atthebaseinfrastructureasaservice(“IaaS”)layer,isfortheretobebilateral tradingbetweencloud intermediaries, inorder to facilitateeffective transfer andhencemanagementof financial risks. Withsuchbilateral trading,often referred toas“OverTheCounter”or“OTC” trading, thereusuallyarisesaprice reporting service thatallows themarketasawhole tohavetransparencyontheexpectedfuturepriceof, inthiscase,cloudcomputingcontracts. OTCmarketsaregenerallyanecessaryprecursortofinancialexchanges,whosepurposeistoreducetransactioncostsforintermediaries’riskmanagementactivities,andminimisecreditrisksthroughtheclearinghousefunction,andtotakeoverandfullystandardisethepricereportingservice.Amarketstructuredinsuchawayisfarless opaque, with far more ability and indeed incentive for all concerned for there to be pricingtransparency.Thiswillunderpinfairpricinginthemarket,leadingtoenhancedtrustintheuseofpubliccloud,andhencegreateruptake.Itiswellworthpointingoutthatthesharedcloudcapacityrunbythemajorcloudprovidershasafarlowercarbonfootprintthanmostorganisations’privateandon-premiseITfacilities. The ability to transfer risk between intermediaries transforms suppliers’ ability to offer priceincentivesinreturnforusageforecasts.Bytransferringtheserisksbetweenintermediaries,theycanbeaggregatedintofirmcommitmentsofthetypefavouredbycloudproviders.Thisreducescloudproviders’capacityplanningrisk,making themmore financiallystable,againpromoting trust in thecloudmarket.Reducingriskallowspricestobelowered,whichcanbepasseddowntobuyers,acceleratinguptake.Thiscreatesavirtuouscyclewithbenefitsforallmarketparticipants.
2 WhyMarketStructureMattersInnovationisalltherage.Fromstartupstolargeenterprises,everyoneistryingtobeinnovative,andcreateproductsthatbringsomethingnewtothemarketthroughsomekindofuniquesellingpoint.Governmentsaredoingtheirbesttofosterthisinnovation,inthehopethattheywillseedthenextgenerationofgloballyrelevantcompaniesthatwilldriveeconomicgrowth,jobs,andanexportsurplus.Themajorityofthesedisruptive companies, small and large, are harnessing on-demand digital technologies, that directly orindirectlyconsumeon-demandcloudservicesthataremassivelyscalable.
Allowingsociety’snextgenerationofcompanies tobeutterly reliantuponasmallgroup,oratworstasingle,underlyingcloudtechnology isnotagood idea. Itwouldbeequivalenttoacountrydecidingtostandardiseuponasinglegenerationtechnologyforgeneratingelectricity.BlackSwanevents,nomatterhowunlikely,cananddohappen,aswasunfortunatelythecaseinJapan,whentherewasatsunamithatimpactedtheFukushimaNuclearPowerPlant.Thatblackswaneventresultedinthelong-termshutdownofeverynuclearpowerplantinJapan,withfarreachingeconomicimpactsforeverypowerconsumerinJapan.However,notethatthemarketstructuredidsupportinteroperability,andwhilsttheremayhavebeensomeinterruption,itwasnotcatastrophic.
Having amarket structure that supports andencouragesdiversificationof such risks is key to avoidingsystemicriskstonotjustNationaleconomies,ortotheEuropeaneconomies,buttotheglobaleconomy.Thecloudcomputingmarketisasingle,interconnectedglobalmarketforsharedITresources,withbuyersandsellersspreadglobally,nodifferenttothemarketfornaturalresourcessuchasoil,coalorevenoflateviaLNG,naturalgas.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 10
Thefirststeptodiversificationistoensurethatthereiscompetitionbetweendifferenttechnologies.Foralong-timeitlookedlikeAmazonWebServices,thepioneerofcloudinfrastructureasaservice,wouldrunawaywiththemarket. That isnolongersoclearlythecase,andinparticularMicrosoft is investingtherequisitebillionsinitsAzurecloudinfrastructureplatform,andisdoingagoodjobofkeepingup.Therearevariousotherswhoarealsoinvestingbillions,mostofthesefromoutsideofEurope,withmixedsuccessinbuildingameaningfulmarketshare.
Thesecondsteptodiversificationofthecloudinfrastructuremarket,istoencouragemultiplecloudserviceproviders toownandoperate thirdpartycloud technology. This is likehavingvariousdifferentpowergeneratingcompaniesallusetechnologyprovidedbyathirdpartywhospecialisesinthetechnology,nottheoperationofthetechnology.Wedoseethisinthecloudinfrastructureasaservicemarket,anotableexamplebeingT-systemsoperatingdatacentresthatruntheAzureStackofferedbyMicrosoft,andtheuseofOpenStackinbothprivateandpublicclouds.
The third step to diversification would ideally be frictionless interoperability between these differenttechnologies,atitsextremeinamannerakintousingapowergridto“mixup”andsharethepowerfromdifferentproviders.Thisiscurrentlystillachallenge,andreallyonlypossiblefortherightusecases,withsufficientexpertiseandsometimesscaletomakeiteconomic.Theauthorsstronglyrecommendthatthissteptowardsdiversificationshouldnotbeachievedbyholdingbacktheleadingcloudprovidersthroughregulatoryactionenforcingstandardisation. Themarketshouldbeallowedtobringforwardcompetingsolutionstosolvetheinteroperabilitychallenges,withasurvivalofthefittestapproachtoselectingthesuccessfulapproaches.Thatbeingsaid,afair,levelplayingfieldneedstobemadeavailabletoinnovativecompaniestryingtocompeteinsolvingthesedifficultchallenges,andthereisclearlyaroleforgovernmentinpolicingthis,throughenforcementofexistingregulation,wherenecessary.TheEuropeanCommissionisalreadyactivelyencouragingandfundingprojectsandinitiativesinthisarea,includingfundingsupportforsuccessfulprojectsthatwanttoachievemarketreadiness,andnotsimplytechnologyreadiness.
RecommendationtotheEuropeanCommission...
...keepfundinginteroperabilityprojects
Giventhattimeisrequiredforthemarkettosolvetheissueofinteroperability,thedoorneedstobeheldopenforfuturemarketentrantstothecloudinfrastructureasaservicemarket.Thebestwaytodothisistoensurethattheasymmetryinsize,andhencebargainingpower,betweenafewhugecloudprovidersand small (in comparison) cloud buyers, which is already well established, does not get any morepronounced.Thewaytodothisistoencourageintermediationbetweenendusercloudbuyersandthecloudproviders,suchthatpurchasingpowerisaggregatedbycloudintermediaries,suchascloudresellersand cloudmanaged serviceproviders.Again leveraging theanalogywithelectricitymarkets, itwas theabilitytoaccesswell-structureddealswithintermediariesinthewholesaleelectricitymarketthatallowedthequitesuddensurgeinmarketshare,andproliferationinnumber,ofindependentelectricitysuppliersintheUK,resultinginmorecompetitivepricingforconsumers.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 11
Figure2Electricitysupplymarketsharesbycompany:Domestic(GB)
Similarly,thereasonwhyspecialistelectricitygeneratorswereabletocompeteintheUKagainstverticallyintegratedcompetitors,wasthroughcarefullystructuredsalesofelectricitytointermediarieswhowereabletomanagesuchlarge,long-termtransactions,withtheassociatedcreditrisks.
Carefullydesignedmarketstructureashighlightedinfigure3,hasafinalbenefitintermsofprovidingapositiveincentiveformarketparticipantstoadoptstandards,oratleastbenchmarkswithstandardwaystomeasureagainst them.This isbecausethereare lowertransactioncosts formanagingpricerisks forproductsthatmatchthemarketstandard.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 12
Figure3Themarketstructureismaturing
RecommendationtotheEuropeanCommission…
...encourageintermediationinthecloudmarket,
sothatspecialistscancompetewiththe
verticallyintegratedcloudproviders
whoselldirecttoenduserbuyers.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 13
3 HowwegothereWhencomputingwasoriginallyofferedtoindustry,itwasasaprivateserviceusingvacuumtube-basedandlatertransistor-basedmainframetechnology.Organisationslargeenoughtoaffordtheirown“privatemainframe”wouldhowevershareitacrossmanylocalusers,andcapacitywasverycarefullyplannedinordertomaximiseutilisationofsuchahugecapitalexpenditure.Whentimesharingwasfirstintroduced,smallerorganisationswouldtakeouttimeshareson“publicmainframes”,andthissharedcomputingmodelpersisteduntiltheintroductionofthe“personalcomputer”.ThePCmadecapacityplanningunnecessarygiven the relatively inexpensive capital outlay for a device that fit under or on a desk and providedprocessing and storage power locally, and generally much faster than could be accessed via thetelecommunications technology of the day. Asmore demanding uses for computing became common,simplermaintenanceneededtobefacilitatedandtheheatandnoisebeingputoutbytheboxunderthedesk became a problem, computer servers were placed in a separate “server room” and the desktopdevices were used as “thin clients” to gain access to the servers - somewhat of a replication of themainframe/terminalmodelemployedbefore.Asserverroomsoutgrewthebuildingstheywerehousedin,andthegrowthineverhigherbandwidthinternetaccess,computerserverswererelocatedtomoredistantanddedicateddatacentres.Thisprovidedtheopportunitytobenefitfromtheeconomiesofsharingagain,withcolocationdatacentresbecomingpopular,sharingair-conditionedbuildingsaslargeasshoppingmalls,butstillwiththeserversthemselvesbeingprivatetotheuser,oratleasttheuser’sorganisation.Itwasalsonotlostonmanythatenergyefficiencywashigheraswell,amajorconsiderationwhen2%ofallpowergeneratedintheUS(andlikelyEU)isforpoweringandcoolingdatacentres.InitiativessuchasaCodeofConductforEnergyEfficiencyinDatacentreshavebeenhelpfulindrivingthistrend.
Theintroductionofvirtualisation,wheresoftwareisusedtocreatetheillusiontoseveralusersofhavingdirect,privateaccesstoaphysicalserver,wheninfacttheyaresharingtheresourcesofthatphysicalserver,dramaticallyshiftedthetrendfurthertowardssharingnotjustdatacentrespace,butthephysicalserversthemselves.The“privatecloud”wasborn,wheremultipleusersfromwithinanorganisationcouldsharethesamephysicalservers,dramaticallyreducingthenumberofserversthathadtobeavailabletoservicethespikyusageprofileofagroupofusers.
Thenin2006,theUSonlineretailerAmazon, launchedAmazonWebServices,a“publiccloud”enablinganyonewithacreditcard,anywhereontheglobe,toprovisioncomputeandstorageinfrastructureofsucha quality thatwithin a few short years, AWS’ public cloudwas puttingmany large enterprise “privateclouds”toshame.Inordertoclassifythecapabilitiesofcompetingpublicandprivateclouds,theNationalInstituteofStandardsandTechnologies(“NIST”)putforwardadefinitionforcloudservicesthathasbeenwidelyadopted.Publiccloudservicesthatarefullycompliantwiththe“EssentialCharacteristics”allowedon-demand,self-serviceaccessavailableoverabroadnetwork,torapidlyelasticresourcestakenfromasharedpool,withapaymentmodelthatdependeduponthemeasuredresourcesconsumed,notsimplythe capacity reserved. Such a definition could equally be applied to electricity and other commoditymarketsfromwhichinsightscanbedrawn.
Inamarketthatrequiresbillionsincapitalexpendituretoachievethehyperscaleneededtocompetewithaglobaloffering,itwasalwaysunlikelythatthecloudinfrastructureasaservicemarketwouldbesharedevenlyacrossalargenumberofcompetingvendors.Theeconomiesofscale,togetherwiththesignificantlead in innovationenjoyed forseveralyearsby thepioneerof themarket,AmazonWebServices,havemeant that only a handful of vendors, notablyMicrosoft, Google and IBM, have really challenged thedominanceofthemarketleader.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 14
However,manyoftherisksassociatedwithhavingamarketdominatedbysofewarebeingmitigatedbyeachvendorrespondingtobotheachother’spricedrops,andeachother’sintroductionofdifferentdealstructures. Furthermore, themarket leader is setting a goodexample to theothers, by supporting anenormousecosystemofresellerandmanagedservicepartner“intermediaries”whoareabletostepintothebillingchainbetweenthecloudproviderandthecustomer,inordertooffertailoredpricing,billingandothermanagedservices.Thesmallercloudvendorsarealsodoingthis,butwithvaryinglevelsofmaturityand success. There are large numbers of European companies, both large and small, who have builtbusinesses as cloud intermediaries, leveraging the advantages they have over the non-European cloudprovidersintermsoflocalknowledge,existingcustomerrelationshipsandnativelanguage,andaddingawidevarietyofvalue-addedservices.
Theresultisthatwhilsttherearealimitednumberofcorecloudtechnologychoicesthatabuyerofcloudinfrastructureservicescanmake,iftheywantamodern,globallyavailable,lowlatencyservicethatfullymeetsNIST’sessentialcharacteristicsfora“cloudservice”,theydohaveamyriadofdifferentsupplierstobuyitfrom,includingEuropeansuppliers.Thesesuppliers,i.e.cloudresellersandcloudmanagedserviceproviderswhoinvoicetheendcustomer,activelycompetetoofferthebestoveralldeal,withvariationsincontractingterms,commitment,paymentterms,financing,billing,currencyoptionsandtechnicalsupport,aswellasofferingtobundleintothedealaplethoraofothertechnicalservices.
The enthusiastic support for this ecosystemof intermediating cloud suppliers is how the largest cloudprovidersaredealingwiththeapparentconflictbetweenaproviderwantingtosellontheirstandardisedpreferred terms, on the one hand, and the buyer needing to buy on terms thatmeet their particularprocurementneeds.Thisisreallyquiteanalogoustothewaythatelectricityisprovidedtothegridbyageneratingcompany(whooftenspecialiseinaparticularpowergenerationtechnology),butthepowerissuppliedandchargedtothecustomerbyasupplier.Itisequallyanalogoustogasandoilmarketswherethewholesalersellsunderverydifferenttermstothewayaretailersellstoendusers.Thismeansthatlessons learnt (oftenthehardway) in thevariousmature,sophisticatedand largely transparentenergymarketsacrossEuropeandaroundtheworld,canbeappliedtotherapidlymaturingcloudinfrastructuremarket. With thisanalogy,wecan thinkof the leadingcloudproviders suchasAmazonWebServices,MicrosoftAzureandGoogleComputePlatformasbeingakintogeneratorswhospecialiseinaparticulargenerationtechnology,withindependentintermediatingsupplierswhobuyfromthesegeneratorsandsell-ontoendcustomers,competingdirectlywiththesupplierbusinessunitownedbythegenerators.Thereisthereforeplentyofcompetitionamongstsuppliers,eventhough,asaconsequenceofeconomiesofscale,generationislimitedtoarelativelysmallnumberofdominantgenerationtechnologiesthatarebecomingthedefactostandardforthemarket.
Aninterestingcomparisoncanbedrawnbetweentherecentstateofthecloudcomputinginfrastructuremarket, and what happened in the US telephone market, when Judge Green threatened AT&T withbreakup,duetoalackofcompetitioncausedbyaverticallyintegratedmonopolyprovider.AT&T,inordertoavoidthethreatenedlossofabsolutecontroloveritssupplychain,insteadvoluntarilygaveupexclusivecontrol of its route to market by selling off the so-called “Baby Bells”. The major cloud providers’enthusiastic support for cloud resellers and cloudmanaged service providers,who act as independentintermediaries between the providers and end customers raises the prospect of another flurry ofentrepreneurialopportunitybeingavailableforcloudintermediaries,Europeanorotherwise,whobuildontopoftheleadingcloudproviders.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 15
AnotherinitiativethatisalsopromisingisbyMicrosoftAzure,whohavepartneredwithT-systems,amajorEuropeansystemsintegrator,toactasDataTrustee,controllingwhohasaccesstocustomers’dataheldinAzuredatacentresbasedinGermany.Thisisnotableasavoluntaryinitiativebyamajorcloudproviderwhoelsewhere ishighlyvertically integrated, todecouple the roleofdesigning thesystemthat“generates”cloudcapacity, from the roleof “operator”, as isnormal in themore“complete”electricitygenerationmarkets.
4 TheRoadinfrontAccordingtowidelyacceptedeconomictheory,marketstendtobehealthier,morecompetitive,andmoreresilient to external shocks, when they are closer to being “complete”. This is an area that has beenresearchedheavilyfollowingtheGlobalFinancialCrisis,albeitcomparisonsbetweenfinancialand“real-world”marketsare fullofpitfalls.A completemarket isone inwhich it ispossible to specialise inoneparticulararea(thusminimisingthebarriertomarketentry),whilstoutsourcingallnon-coreriskstothirdparties, at fair, rational and transparent prices. Markets that approach completeness may well havevertically integrated participants, but the market is structured such that specialists are able to thrivealongside them.Where the market is not structured in this way, for example in Germany where thevertically integrated Deutsche Telekom owns “the last mile” of telecoms connectivity to residentialproperties,competitionsuffers.
“Theverticallyintegratedgiantsofthecomputerindustry,firmssuchasIBM,DigitalandBurroughs,werefelledlikeyoungsaplingswhenattheendofthe1970sAppleformedanetworkofindependentspecialiststhatproducedmachinesfarmoreefficientlythanthedo-it-allgiants.”-TheEconomist
Asexplainedabove,theleadingcloudprovidershaveenabledaplethoraofintermediariestospecialiseinsupplyingthecloudproviders’servicestotheendcustomer.So,atleastfromaprocurementperspective,thecloudbuyerdoeshavealotofchoice.InBox1,weexplainthedifferentwaysinwhichacloudbuyercanpurchaseinfrastructurecloudservices.
Box1–HowtoBuyCloudonYourTerms
Direct: Buying directly from the Cloud Provider is the most obvious approach, and is the moststraightforwardprovidedtherearenodifferencesbetweenthestandardisedway inwhich theCloudProviderwishestoselltoyou,andthewayinwhichyou,thebuyer,wishtomakethepurchase.However,themomentyouneedanythingnon-standard,itisworthconsideringtheotherchoicesbelow.
Reseller:ACloudResellerhasaverystrictdefinition.ItmeansacompanywhowillresellcloudservicesinamannerapprovedbytheCloudProvider,whohasengagedtheResellertoeffectivelyactasitsproxy,inordertocontractwithnon-standardcustomers.Resellersgenerallyreceiveonlyasmallrebatefromthe Cloud Provider for providing an outsourced negotiation service, and are expected to achieveprofitabilitybyaddingvaluetothecustomerinotherways.However,usingaResellerisnottheonlywaytoplaceathirdpartyintothebillingchaininordertogetatailoreddeal.AManagedServiceProvider(“MSP”)canbeusedforthistoo.
ManagedServiceProvider:CloudProviderstreatManagedServiceProvidersascustomers,nodifferenttoanyenterprisecustomer.ManagedServiceProvidersareallowedtoprovideaccesstocloudprovideraccountsthattheynominally“own”,butwhichinpracticemaybeusedexclusivelyforthebenefitofa
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 16
particularcustomeroftheManagedServiceProvider.Thisshouldnotbeconfusedwiththemajorcloudvendors’definitionof“resale”,howeverallthebenefitsofhavingathirdpartyinthebillingchainwhocanintermediatebetweenthetradingpreferencesofthecloudproviderontheonehand,andthecloudbuyerontheother,arethesameaswhenbuyingthroughareseller,withtheaddedadvantageoftheMSP being far less restrained than a reseller. There are even examples of where a billing chain iscomposedofalargeResellerfollowedbyasmall,moreinnovativeManagedServiceProvider.
Independentcloudsuppliers(i.e.cloudintermediarieswhoselldirectlytotheenduserbuyers)arelikelyto grow theirmarket share, firstly because end customers often do prefer the tailored services that aspecialistcanoffer,butalsobecauseitisintheinterestsofthebiggestcloudproviderstoencouragethis,certainly in Europe. There is a European Directive that places particular obligations on any marketparticipantdeemedtocontrolmorethan30%ofanygivenmarket.Suchadominantvendormustbeverycarefulinapplyingverticalrestraintsonits“resellers”,wherethisuseofthetermmaybebroaderthanthatused(inapoorlydefinedmanner)inthecloudmarket,andassuchcouldincludeintermediariesotherthanCloudResellerssuchasCloudManagedServiceProviders,althoughsomerecentcontractualupdatesmakethisunclear.Theleadingcloudprovidersarereallyinadifficultposition,astherearegreatadvantagesforeveryoneinhavingago-to-marketapproachthatisconsistent,logicalandensuresthatasmanybuyersareservicedaspossible,byleveragingoutsidehelpinservicingcustomersandprospects.However,gotoofarwiththeorchestrationofresellers,andthemajorcloudproviderscouldbeaccusedofundesirablemarketpractices, which is clearly not their intention, at least in the experience of the authors and thoseinterviewedoverthelast2years.ThisareaisofparticularconcerntoAWS,whoiswidelyassumedtobeabovethe30%applicabilitythreshold,andmayberelevanttoMicrosofttoo.
Inmarketsthatdosupportbothverticallyintegratedparticipants,whobothownandoperatefacilities,andselldirecttocustomers;andspecialistparticipants,thereisabigissuearoundtransferpricing.Forexample,intheelectricitymarkets,itisverycommonforverticallyintegratedcompaniestoclaimthattheirsupplycompanies,whoselldirecttotheircustomers,sometimesunderarangeofbrands,makevirtuallynoprofit,therebyimplyingthattheirpricingishighlycompetitiveandhencefair.Theaccusationfrequentlylevelledat these companies is that the transfer pricing of sales from the generation part of the business thatprovidesthepower,tothesupplysideofthebusinessthatsellsittothecustomers,isdeliberatelysethighenoughtowipeoutallprofitinthesupplycompany.i.e.itisthegeneratorthatreportsalltheprofits.Themostcommonwaytoavoidthisisfortransferpricingtobelinkedtothepublishedpricingfromwholesaletradingamongstintermediaries,aswegoontodescribeinthenextsection.
5 MakingthecaseforCloudTradingAsmentionedabove,thenextstageinmarketdevelopmentistoenablebilateraltradingbetweencloudintermediaries.Suchtradingservesavaluablepurpose-itallowsintermediariestooffloadriskstheyprefernottohold,totheirpeers,atanegotiatedprice,aswellaslimitingthemarketpoweroftheprimarysellers.Ifanintermediarycanknowthepriceatwhichtheycanoffloadariskydeal,thentheywillbefarmorewillingtotakeonthatriskfromacloudbuyerorcloudproviderwhowouldotherwisehavetokeeptherisk.Box2goesthroughaworkedexampleofhowOTCtradingdeliversbenefitsforacloudprovider,andBox3foracloudbuyer.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 17
Box2–HowOTCTradingHelpstheCloudProvider
Acloudprovidercreatesariskwhentheybuildanewdatacentre,leasemorecollocationspace,orevenjust buy a new rack of servers. They pay or commit a certain value, and if they do not recoup thatinvestmentoverthelifetimeoftheasset,theywill losemoney.Iftheywishtoborrowmoneyfromabanktofinancetheinvestment,thebankwouldtypicallyaskforacontractofsalethatshowsthefuturecashflowsthatwillrepaytheloan.Thewaycloudproviderscurrentlyfinancetheirexpansionisnotdonelikethis,withthesmallestprovidershavingtoraiseequitytofundgrowth.Thetrouble is thatwhilstmanycloudprovidersdoofferfixedpricedealsthatcoverthefull3-yearlifeofaserverrack,cloudbuyersbuyrelativelylittleofwhattheyuseontheselong-termdeals.Thecloudbuyershavebeensoldontheideathatthepubliccloudisallaboutelasticityandon-demandbursting,ratherthanaboutlong-termcapacity planning,which is all true, but this flexibility is included in the on-demandprice at a 400%premium.
Therefore,ratherthanrelyingsolelyonsellingtoendusercloudbuyers,primarilyaton-demandpricing,atcertaintimesitisfarbetterforacloudprovidertobeabletosellalargevolumeofcapacitytoacloudintermediaryundera long-termdeal, ata fixedprice,andevenprepaid.This thenallows lowercostfinancingofthecloudprovider’sgrowth,particularlyforsmaller,newentrantcloudproviders.Ofcourse,inorderforthecloudintermediarytobeabletodealwiththatrisk,theyneedtobeabletobreakitupandselliton“vertically”toitsowncloudbuyers,andalso“horizontally”toothercloudintermediarieswhomaybewillingtoimmediatelytakeonashareoftheriskataprice.Itisclearthateventhelargestandbestfundedcloudproviderswouldliketobeabletoselllong-termdeals,asAmazonWebServiceshasoffered“heavyutilisationReserved Instances” for several years, andGoogle recentlyannouncedlong-term“CommittedUseDiscounts”.
Box3–HowOTCTradingHelpstheCloudBuyer
Theusageofcloudresourcesbymostcloudbuyersisnottotallyadhoc.Beneaththeautoscalingusage,therearesomevirtualmachinesanddatabasesthataredeliberatelyleftrunninglong-term,withgoodforecastability as to size, operating system and location. There are then otherswhose use is betterdescribedas“persistent”thannecessarilyas“forecastable”.Enterprisestendtobesubjecttoalotofinertia,andwhenyouaskadeveloperthechanceofaparticularmigrationhappeningwithinacertainperiodyoucanoftenextractaprobabilisticforecastfortheproject.Withtherightfinancialincentivestohand, a cloud supplier who has a close relationship with its cloud buyer, is able to collate theseprobabilisticforecastsandconvertthem,givenareasonableattitudetorisk,intolong-termpurchasesthatmatcheitherhowthecloudproviders liketosell,orshortercontractsthatcouldbeavailablebytradingbilaterallywithothercloudintermediaries.Thecloudbuyeristhennotonlyhelpedbythecloudsuppliertothinkaboutitsfuturecloudusage,butisalsogivenalowerpricethanthedefaulton-demandpricing.
For the sophisticated cloud buyer, who has invested in technical systems that allow their cloudinfrastructuretobeprovisionedinmultipledifferentwaysgivensufficientnotice,theycanaccessevenbetter pricing from a cloud intermediary, by helping to manage the capacity planning risk of theintermediary.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 18
SuchOTCtradingisnotfardowntheroadahead.Severalglobalcloudintermediaries,headquarteredinEuropeandelsewhere,haveexpressedinterestinmanagingtheircapacityplanningrisksthroughbilateraltradingunderanOTCmarketstructureandtheauthorsareworkinghardtoconvertthisintoareality.
6 EuropeanInitiativespavingthewayThissectionoftheroadmaphighlightsseveralareaswhereEuropeaninitiativesarehelpingtoguide,bolsterandreinforcethestructureofthecloudmarkettowardsonethatencourageshealthycompetition.Thesearespecialistframeworksandservicesthataddressveryspecificmarketrequirements,thatremovefrictionintheadoptionofcloudservices.Weincludethis,aswebelieveitisimportanttohighlightareaswheretheEuropeanCommission’sprioractionshavebeenhelpful,ratherthansimplyaskingfornewactions.
6.1 DigitalSingleMarket
ThefragmentednatureoftheEuropeanUnionfromaregulatoryperspective,isdeeplyunhelpfultohome-grownEuropeancompaniestryingtocompetewithglobalcompetition,particularthatfromtheUSA.ASiliconValleystartupcanspecialise inatinynichedigitalservice,andsellthat inastandardwaytoanycustomeracrossthehugeUSmarket,withessentiallynofriction. Thisallowsthemtorapidlyachieveascalethatthenpermitsinternationalexpansion,andcompetitioninaglobalmarket,withalltheattendantregulatoryandprocessbarriers.Incontrast,aEuropeancompany,withasimilaridea,cancurrentlyonlytarget equivalent niche customers for its particular specialist digital service, in its own country. Themomenttheytrytosellintoevenaneighbouringcountry,inadditiontoinevitablelanguagebarriers,theyhavetocontendwithdifferencesinregulatoryapproachandbureaucracy.WorkingtowardsaDigitalSingleMarketisahugelyworthwhileaim,bringingenormousbenefitstoallwhohaveaccesstoit.
Oneofthewaysinwhichthecloudcomputingmarkethasnotyetcaughtupwiththeelectricitymarketsisintheareaofstandardsandinteroperability.Auserofelectricityistotallyunawareofwhatblendofpowergeneration technologies is being used to deliver its needs, because the electrical (transport) grid sitsbetween the generator and the consumer, effectivelymixing up the electricity generated by differentproviders,usingdifferent technology.The reasonwhy thisworks such that supplyanddemand remainbalancedatallpointsonastablegrid isduetoacombinationofstandards,andcarefulmarketdesign:injectionintothegridissubjecttostrictstandardsofelectricalpower,frequencyandvoltage;andthereisacarefullyorchestratedshort-termcapacitymarket.Whilsttheinternetmayappeartobeequivalenttothepowergridintermsofprovidingaccesstocapacity,theinternetdoesnotforcecooperationinstandardsandmarketdesign,allowinghighlygranularcontrolofwhichuserscanaccesswhichcapacity.Ontheonehand,thisisahugeadvantage,butthatadvantagecomesatacosttothelevelofcompetitioninthemarket,asitleadstofragmentedmarketswherecloudresourcesmaybetradedinprovider/generatorsilos,butnot across providers/generators in a market that spans all. This effectively commoditises users intofungiblesourcesofrevenueforthecloudproviders,butinhibitstheusersfromtreatingthecloudprovidersasinterchangeable.
Europeancloudcomputinginitiativeshavelongproventhattechnically,API-levelaccesstoIaaSresourcesacross resource providers is technically achievable, feasible and adds value for the cloud resourceconsumersacrossawidevarietyofsignificantusecases,wheresufficientlysupportedbytheproviders.However,thereremainchallengesinmakingthisworkacrossallcloudproviders.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 19
Byway of example, Oliver Tearne, a seniormathematicalmodeller at theUK’s Animal& Plant HealthAgency,wishedtousethepubliccloudtoaddelasticburstableinfrastructuretotheirlimitedon-premisescapabilities.
“Initiallywetriedtoabstractourselvesfromtheunderlyingcloudproviders,byprovisioningviaacommonAPI. Wewantedtoavoidvendor lock-in,andfeltthatforourfairlysimpleusecases itshouldactuallybeeasier.Wefoundthatthetechnicalchallengesofdoingthiswerestilltoobigtojustifytheeffortforthescalethatwearecurrentlyat,sowedecidedtointerfacedirectlywiththecloud provider, whilstmaintaining the ability to switch funds from one provider to another byprocuringthroughafinancialcloudbroker.”
Suchfrustrationswithbeingcloudagnosticatsmallscaleswithparticularprovidersareunlikelytoremainthecaseindefinitely,as it iscertainlyalreadypossiblewithsufficienteffortfortherightusecases,withsufficientlycompliantcloudproviders.
Asforelectricity,itwillbeacombinationofcarefullyagreedstandards(oratleastbenchmarkmetrics),andmarketstructurethatwillacceleratethistrend.TheEuropeanCommissionhasshownsteadfastsupportforinitiativestodevelopcloudstandards,suchasthoselistedhere,andtoencouragefederationsofsmallerclouds,suchasEGIforresearchers.ThistrendisverylikelytoacceleratewiththeEUGDPRcomingintoforce inMarch2018,whichenshrinesconsumersrights to (personal)dataportability into itsarticle20.Already,theCommissionthroughcommissioningstudiessuchasSMART 2016/0032, explores expanding this issue to general data portability towards switching cloud providers in the same or similar fashion as switching energy suppliers.
It is now important that the Commission also supports more sophisticated market structure, as to be a success, the two need to go hand-in-hand.
RecommendationtotheEuropeanCommission…
...encourageremovalofanybarriersto(voluntary)abstractionoftheuserexperiencefromtheservicesoftheunderlyingcloudprovider
6.2 EuropeanOpenScienceCloud
TheEuropeanOpenScienceCloud(“EOSC”)isahighprofileEuropeanCommissionfundedefforttoensurethatEuropeansciencehasaccess to therequisitecloud infrastructureresources tomaintain its leadingglobalpositioninscientificresearchanddevelopment.ItplanstointerconnectthroughaEuropeanDataInfrastructure, existing research infrastructures across Europe allowing 1.7 million researchers and 70millionscienceandtechnologyprofessionalstheabilitytoaccessandprocesslargedatasetsstoredinthecloudinamannerthatisFindable,Accessible,InteroperableandRe-usable(“FAIR”).
TheEOSC’sHighLevelExpertGrouphasrecommendedframingtheEOSCastheEUcontributiontoafuture,globalInternetofFAIRDataandServicesunderpinnedbyopenprotocols.Theyhavealsohighlightedthatthe largermarketopportunity liesclosesttothebusinessapplication. Theyestimatethathalfamillion“coredatascientists”areneededtomakethemostofopenresearchdatainEurope.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 20
ItishugelyencouragingthattheEuropeanCommission’sfocusisonenablingtheapplicationsthatcloudfacilitates, rather than on the cloud infrastructure itself. In the sameway that on-demand electricitycatalysedinnovationandbusinessgrowthinheavyindustry,publiccloudserviceswillcatalyseinnovationandbusinessgrowthintheapplicationstheysupport,manyofwhichwilldoubtlesscomeoutofscientificR&D.
“ResearchInfrastructure’slong-termneedsaremeasuredindecadesandhenceastableandwell-structuredcloudservicesmarketisessential.”-DrBobJones,CERN
6.3 ProcurementInnovationforCloudServicesforEurope(PICSE)
Theelasticityandon-demandaccessofthepubliccloud,hasbeenbothaboonandabaneforprocurementprofessionals.Itisamazingtobeabletomeetunforecastorganisationalcomputingdemandsbyprocuringon-demandpubliccloudresources.Atthesametime,aconsumption-basedpricingmodelsimplydoesnotfit well into inflexible procurement systems that expect a fixed price for a fixed amount of a fixeddeliverable.ThePICSEprojectwasfundedbytheEuropeanCommissiontotrytofindinnovatewaystomanagetheseprocurementchallenges.Atthesametime,leadingcloudprovidershavebeentacklingthesamechallengesbytryingtobemoreaccommodatingtotheneedsofthepublicsector,settingupspecialistteamsforpublicsectorsales,andsupportingcloudresellersandmanagedserviceproviderswhohavebuiltacorecompetencyofintermediatingbetweentheseconflictingtradingpreferences.
Deliveringprojectsthatfindinnovativewaystoaddresstheseneedswithtechnologythatisdeemedtobeready,i.eithasahigh“TechnologyReadinessLevel”,isallverywell,butunlessitisalso“marketready”,itwillneverhaveanimpact.ItisverypleasingtoseethattheEuropeanCommissionfunded,anditsprojectsembraced,otherworkdonebyCloudWATCH2thatbroughtforwardacombinedframeworkforMarket&TechnologyReadinessLevels.
RecommendationtotheEuropeanCommission…
...ensurethatsuccessfulEC-fundedprojectsachieve
market-readiness,notsimplytechnologyreadiness.
7 PotholestoAvoidThejourneytowardsahealthy,competitivecloudmarketthatistrusted,reliableandsustainable,isunlikelyto be a smooth one. Most other more mature markets that bear some resemblance to the cloudinfrastructuremarkethavehadtheirfairshareofbumpsontheirjourneytowardsmarketsophistication,anditisgenerallysensibletotrytolearnfrompriormistakes,toavoidrepeatingthem.Below,wehavelistedanumberof“potholes”thatshouldbeavoidedasEuropecontinuesonitscloudjourney.
7.1 OptimalStandardisation
Standardisationandinnovationarebothpositiveattributes,andyetinmanywaystheyaretheoppositeofone another. Judicious application of widely accepted standards can remove unnecessary variability,drivingdownoperationalcostsassociatedwithsupportingmultiplevariations,andprovidingaplatformfor
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 21
valuablefurtherinnovation.Overlyenthusiasticenforcementofimmaturestandards,ontheotherhand,canmakeinnovationimpossible.Thecloudmarketparticipantsmustcollectivelyjudgetherightpaceatwhichtostandardisethetechnologyandcontractualelementsthatmakeupthemarket,inordertoachievetheoptimalbalancebetweenthesetwopositives.Itisalsoworthpointingoutthatifthereisaflawinaparticularstandard,andithasbeenadoptedinallcases,thenitrepresentsasinglepointoffailure.Inmanycases,itisbettertohaveatleast2standardsthatareavailableforadoption,inordertodiversifythisrisk.Standardisation onto a single standard that turns out to be fundamentally flawed could cause enoughdamagetoconsumertrustincloudastobringourcloudjourneytoagrindinghalt.
7.2 Withgreatpowercomesgreatresponsibility
Anunusualfacetofbeingacloudprovider,ishowmanyadditionalservicescanbeincrementallybuiltoutonceyoucanoffer theunderlyingcorecompute, storageandnetworkingcapabilities inamanner thatmeetsNIST’sessentialcloudcharacteristics.Withoutofferinganyservicethatlookedlikeanythingotherthanalogicalextensionoftheircorebusiness,severalofthemajorcloudprovidershavegonefromofferinghourly rental of virtual servers, to offering to make on-demand predictions about their customer’sorganisation by automatically building machine learning algorithms that interrogate and process vastamountsofyourorganisation’sbigdata.
It has been observed by commentators such as the Economist that the major cloud providers lookremarkably like conglomerates. It is usual for investors to apply what is known as a “conglomeratediscount”whenputtingavalueonthesharesofcompanieswhorunamultitudeofdifferentbusinesses,i.e.conglomerates.Thediscountisattributedtoahostofjustificationssuchasinefficienciesduetoalackofspecialisation,andtheriskthattopmanagementcannotbefocussedeverywhereatonce.Ithasbeenobservedthattheconglomeratediscountdoesnotappeartobebeingappliedinthiscase,whichimpliesthatinvestorsmaybeignoringtherisk,andjudgethatthecloudproviderscankeeponrunningsomanydifferentspecialismsinparallel,withoutfearofmishap.Thesemajorcloudprovidersareresponsibleforsupportingthemajorityoftheworld’sdigitalbusinesses,anditiscriticalthattheycontinuetokeeptheircorebusinessstable.OneexampleofaconglomeratewhorunsanIThostingbusiness,inthiscasea“privatecloud” for their online gamers is Sony. They suffered a “Black Swan” event, which caused weeks ofdisruptiontotheirPlaystationNetworkcustomers,whohadnoothermeansofplayingthoseparticulargameswithanotherprovider.Theirbusinessprocesseswerenotassophisticatedasotherlessdiversifiedcompanies, such as Microsoft, who had implemented 2-factor authentication 3 years earlier for thecompetingXboxonlineplatform.“Itwasthelargestsecuritybreachofitskindtoeverhitconsolegamers,andaneventwithhugerepercussionsforPlayStation-bothintheshorttermforitsusers,leftforweekswithoutaccesstoonlineservices,andlongertermasSonysoughttowinbackcustomertrust.”-TomPhillips,Eurogamer.
TheimpactofanITprocessfailureisnotlimitedtoonlinegamers.TaketherecentBritishAirwaysexampleof where a “computer glitch” has caused a systems failure. Unlike other recent examples affectingLufthansa andAir France,where “glitch” is arguably a fair description, given the short duration of thesystemsfailure,theBritishAirwaysfailurecausedaglobaloutageofawholeswatheofBAsystems.Thisresulted in thegroundingofall flightsglobally toand fromHeathrowandGatwick foroveraday,withknock-ondisruptionfeltineveryairportthatBAfliesto.
InvestorsinIAG(theparentcompanyofBAandIberia)wouldcertainlyhaveseenthis“computerglitch”asapotholeintheirrunway,assharepricesdroppedby4%inLondonand2.8%inMadrid,asaresult.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 22
Thepointwearemakinghere is thatwhere companies grow to the kindof sizewherea failurehas anoticeable impactonsocietyasawhole, theyshouldhavea responsibility toworryabout these“BlackSwan”risksandimplementprocedurestomitigatetheeffects.Thisisgenerallysomethingthatrequiresthe input of senior management, whose attention should not be spread too thinly across differentbusinesses.
Imagine how much worse such a global outage could be for consumers, if it happened to the coreinfrastructureservicesofamajor IaaSproviderdisruptinga largenumberoftheirdiversifiedcustomerswhoprovideservicesrangingfromreal-timeairlinearrivaltimestotrainbookingstoweatherreportstovacationplanningtosupermarketdeliveriestoschedulingGPappointments?
7.3 Rebalancingtheasymmetryinmarketinfluence
Atpresent,themajorcloudprovidersdefinetheirservices,howtheyarepriced,andthetermsunderwhichtheyaresold.There isnegligiblenegotiationwitheventhe largestbuyers,whomustsimplyaccepttheterms,oruseaninferiorservice.Thisisn’ttoounreasonable,untilyouaddinthefactthatthecloudproviderhaslikelyincludedinitstermsandconditionsthatitcanunilaterallychangethosetermsandconditionswhenevertheywant.
Most other markets either have less pronounced asymmetry in how buyers and sellers influence themarket,orthereismorenegotiation(withallthehighertransactioncoststhatthatentails)andinregulatedmarketsthereisconsiderablymoreconsultation,withmajorbuyersbeingrepresentedbysomeformofindustrybody, suchas theMajor EnergyUsersCouncil, that represents25%of all theenergyused forindustrial purposes in the UK, an example at national level. At a supranational level, IFIEC Europe“representstheinterestsofindustrialenergyusersinEuropeforwhomenergyisasignificantcomponentofproductioncostsandakeyforcompetitiveness intheiractivities inbothEuropeandthroughouttheworld.”.Thereisevenaglobalequivalent,IFIECWorld,aNon-GovernmentalOrganisationrecognisedbytheUN.
ItisverytemptingtoparaphraseIFIECEurope’smissionstatementforsomefutureassociationofmajorcloudusersundertheDigitalSingleMarket,byswapping“energy”for“cloud”:“AnInternationalFederationofIndustrialCloudConsumersshouldbefoundedonthebeliefthatcompetitivecloudsupply,responsibleuse of cloud and consumer choice and flexibility, are the necessary ground rules for competitive andsustainableindustrialactivityinEurope.”
Thecloudmarketisalreadyinthisparticularpothole,astherearecontractualtermstechnicallyinforcetodaythataresoinappropriateastolikelybeunenforceable.“Unlikely”however,isnotenoughtocomfortrisk-averse potential users. Unless a better approach tomanaging changes to terms and conditions isadopted,werun the riskof falling into similarpotholes in the future.An industryassociation forcloudbuyerscouldhelprepresenttheinterestsoftheconsumer.
RecommendationtotheEuropeanCommission...
...support(withfunding)anywell-organisedindustrybodiesthatrepresenttheinterestsofbuyersandintermediaries,andareindependentofthedominantcloudproviders.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 23
8 RecommendationsfortheEuropeanCommissionTheauthorswouldliketoputforwardthefollowingtwolistsofrecommendations,onebeingpoliciestoencourage,theothertoavoid.
Encourage:
1. Publicsectorprocurementofcloudthroughintermediariessuchascloudresellers,managedserviceproviders,orevenpublicsectorcooperatives,inordertoaggregatebuyers’purchasingpower,whilstkeepingtailoredprocurementterms.
2. Even-handedtreatmentofeachclassofcloudintermediarybycloudproviders,inordertomaximisecompetitioninthecloudsupplymarket,maximisingbuyerchoice.
3. Innovationinhowtofacilitateswitchingbetweencloudsolutions,andhowtoabstracttheuserexperienceawayfromtheunderlyingcloudprovider.
4. Uptakeofstandards,wherecommonalitybringseconomicadvantagethroughincreasedsharingandcompetition.
Avoid:
1. DistortingmarketforcesbyprovidingstateaidtoaEuropean“cloudchampion”2. Imposingstandardswherethatwouldinhibitinnovation3. Disadvantaginglocalcloudbuyers,bycreatinglocalrules4. Repeatingthemistakesalreadymadeinotherrelatedmarkets,bylookingforanalogouspast
situationsinothermarkets.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 24
9 MethodologyforResearchingthisRoadmapMuchoftheresearchconductedforthisRoadmapwascarriedoutintheformofinformal,non-attributable,off-the-record interviews. This approach was found to be necessary as we found there to be a starkdifference between views expressed privately, and what could be cited and attributed to particularindividualsatparticularcompanies.Wehavegoneoutofourwaytorespecttheanonymityofthosepeoplewhowerekindenoughtosharetheirhonestviewsandconcernsregardingthestateandfutureofthecloudmarket.Wecanhoweverlistthetypesoforganisationswehaveinterviewed,inordertogiveaflavourforthebreadthofinterestedparties:
● US“hyperscale”IaaSpubliccloudvendors● EUmultinationalprivatecloudvendors● EUsmallerscaleIaaSpublicandprivatecloudvendors● EUandUSdatacentreoperatorsandcollocationproviders● Technologyproviderstopubliccloudproviders● EUregulatoryagenciesandauthorities● US&EUendusercustomers,bothcommercialandpublicsector,largeandsmall.● Lawfirmsspecialisinginmarketregulation● Otherexpertsinmarketevolutionandinnovation
Wheretheauthorshaveleveragedthebenefitofyearsofexperienceinanalogousmarkets,wehavesoughtto back up any analogies by seeking support from the interviewees above, and/or by finding suitablereferences.
The development of the content for this Roadmap went throughmultiple iterations, and we sourcedfeedbackateachstage.Westartedoffwithadeliberatelyprovocativedraft,highlightingmanyoftheBlackSwan risks that could be envisioned for the market, and circulated it to selected industry figures,confidentiallyduetoitsprovocativecontent,inordertoassesshowwidelyanddeepBlackSwanriskswerefeltacrossthemarket.Wepresentedourearlythoughtsandsourcedmoregeneralfeedbackatvariousconferences,butmostnotablyatHelixNebula7thGeneralAssembly,whereJohnWoodleygaveakeynotespeech.Atthatpoint,itwasdecidedtodelaydecisionsastowhethertopublishsomeofthecontentuntilthisfinalversion,asseveraloftheissueswereinastateofflux,anditwasfeltthatabetteroutcomemightbemorelikelyiftheissueswerenotfurtherpublicisedatthattime.Basedonthefeedbackfromvariousexperts,someofwhomhadadvisedtheEuropeanCommissionpreviouslyonrelatedissues,wepreparedthePreliminaryVersionoftheRoadmap,whichwaspublishedinApril2016,followingconsultationwiththeEuropeanCommissionandcertainregulatoryagencies.
FollowingtheissueofthePreliminaryVersionwehavecontinuedtoconsultwidelyonthissubject.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 25
10 ConclusionInnovationinthecloudcomputingmarkethasbeen,andcontinuestobe,conductedatbreakneckpace,witheveryincrementalservicelayeraddedontop,andeveryfeatureenhancedaroundthesides,providingevergreatervalueandconveniencetotheuserofthecloudsolution.Thisfreneticpaceofdevelopment,alongwithacompletelackof“official”marketstructureholdsboththepromiseofcontinuedunfetteredinnovation,andtheriskofmishapasaresultofstructuralmarketinstabilitiesthatposesystemicmarketrisks.
However,themarketdoesseem,inmanyareas,tobeself-regulatinginafairlyhealthyway.Supportforcloud resellers, cloud managed service providers and other cloud intermediaries by the largest cloudproviders is largely healthy,with good support for the natural conflict that occurswhen indirect salesoperateinparallelwithverticallyintegrateddirectsales.Thisgivesthecloudbuyerplentyofchoicearoundwhotobuycloudservicesfrom,andthetermsonwhichtomakethatpurchase,evenifbehindthescenes,theunderlyingtechnologychoiceisdominatedbythoseathyperscale.
The authors recommend that any regulatory effort is focussed on ensuring that the current level andfairnessofcompetitionpersistsattheinterfacewiththecloudbuyer,suchthatbuyersretainchoiceandeaseofswitchingtheircloudsupplier.Ascloudintermediariesbuildmarketshareinsupplyingpubliccloudservicestoendbuyers,thecurrentmarketasymmetrybetweenhyperscalesellers,andcomparativelytinybuyersmayreduce,balancingtheirrespectivenegotiatingpositions.Theexistenceofcloudintermediaries,particularly if theyareable to trade riskamongst themselves,will leave thedooropen to futurecloudproviders,whoaremorelikelytoenterthemarketbyspecialisinginaninnovativenewmeansofprovidingcloud services than by competing on price for an inappropriately standardized “lowest commondenominator”cloudservice,againstthosewithhyperscaleeconomies.
www.cloudwatchhub.eu|@CloudWatchHub
D4.3ROADMAPTOACLOUDMARKETSTRUCTUREENCOURAGINGTRANSPARENTCLOUDPRICING–FINALITERATION 26
LogTableDOCUMENTITERATIONS
V1 Version1ofdocument John Woodley, James Mitchell & Frank KhanSullivan,StrategicBlue
V2 Internalreview NicholasFerguson&SilvanaMuscella,Trust-IT;DavidWallom&MichelDrescher,UOXF
V3 PMBreview NicolaFranchetto,ICTL;MarinaBregu,CSA
VFinal Finalversion JamesMitchell,StrategicBlue