CON6757 - Oracle · 36 • Epsilon is all-encompassing global marketing company, we are global leader in turning data-driven marketing into lasting relationships

Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|

CON6757ImprovingApplica/onCon/nuity

CarolColrainTechnicalLeadforClient-Failover,RACDevelopmentGairikChakrabortySeniorDirector,Database,EpsilonTroyAnthonyCloudEvangelist,RACDevelopmentMichaelTimpanaro-PerroRaSeniorDirectorProductManagement,RACDevelopment


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirecUon.ItisintendedforinformaUonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncUonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andUmingofanyfeaturesorfuncUonalitydescribedforOracle’sproductsremainsatthesolediscreUonofOracle.

3


• Availablenow– ExadataExpressCloudService

• Comingsoon– DatabaseCloudServices– ExadataCloudMachine

4

AnnouncingOracleDatabase12cRelease2onOracleCloud

OracleispresenUngfeaturesforOracleDatabase12cRelease2onOracleCloud.WewillannounceavailabilityoftheOn-PremreleasesomeUmea\erOpenWorld.


ProgramAgenda

ProblemsFaced

Areyourbuildingblocksinplace?

EasystepsforApplicaUonConUnuity

Epsilon-ProducUonImplementaUon

1

2

3

4

5

Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.| 6

Whatproblemsfaceusersatdatabaseoutages?

1


Databaseoutagescauselossofin-flightwork,leavingusersandapplicaUonsin-doubt

–  RestartapplicaUonsandmid-Uers

–  UserfrustraUon–  Cancelledwork–  Duplicatesubmissions

–  Developerpains–  Errorsatscheduledmaintenance

Pre-12cSitua/on

7

Sorry.InternalServerError-500ErrorWearecurrentlyexperiencinganissuewithourserversoncoolcar.com.Pleasecomebacklater.

In-FlightWork


HowdowesolveforallapplicaUons?

TransparenttoapplicaUonsandmid-Uers

ScalestointernetandcloudoperaUons

CoverseverydriverandmostapplicaUonfeatures

Sameeverywhere–connecUng,failover,interfaces

TakesHAadopUonoutofthedevelopershandstooperaUononly

8


2

Areyourbuildingblocksinplace?


Run/meAffinity.FailoverwithinandacrossSites.

10

AcUveDataGuard– Scheduledswitchover– DataProtec3on,DR– QueryOffload

DataGuard– Scheduledswitchover– DataProtec3on,DR

GoldenGate–  Scheduledswitchover–  Ac3ve-ac3vereplica3on–  HeterogeneousSharding– MassiveOLTP–  Scheduledswitchover–  Ac3ve-ac3vereplica3on–  Heterogeneous

Replicas

RAC– OnlineRollingMaintenance– Scalability– ServerHA

RACOne– OnlineRollingMaintenance– ServerHA

ProducUonSite


12cR2OracleRealApplica/onClustersCon/nuousServiceAvailability

RealApplica/onServiceLevels

• ScalesPDBsandServices

• 2seconddetecUononEXA

• Recoveryinlowseconds

• Drainsworkgradually

• Recoversin-flightwithAC

“AlwaysRunning”

11


Recoverin-flightwithApplicaUonConUnuity

ADGsessionssurvivestandbyrolechange

Drainthenswitchover,ACrecoversstragglers

Switchover to <db_resource_name> [wait]

FAILOVER

DataGuardObserver

RACPrimary RACStandbySiteA SiteB

12cR2OracleAc/veDataGuardCon/nuousServiceAvailability

12


CDB1

GL AP

CDB2

OEOE OE

FinalREDOApply

PO

TNSLISTENER

TNSCOMMONLISTENERREGISTRATION

RelocateandDrainReadserviceahead

OnlinePDBrelocateDrainandcutoverinunder10seconds

12cR2OracleMul/tenant

Applica/onCon/nuitymovesstragglers

Con/nuousServiceAvailability

13


UseServicesforLoca/onTransparencyServicesprovidea“dialinnumber”foryourapplicaUon

• RegardlessoflocaUon,applicaUonkeepsthename

• Moving,reshaping,prioriUzingcontrolshowaserviceisoffered

• BatchandOLTPseparated

• DBandPDBnamesforadminonly

Node 1

RACinstance

Node 2

RACinstance

OLTPservice

Batchservice

14


Areyourconnec/onsproperlyconfigured?12cR2SameforAllDrivers

alias=(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=20)(RETRY_DELAY=3)

(TRANSPORT_CONNECT_TIMEOUT=3)(ADDRESS_LIST= (LOAD_BALANCE=on) (ADDRESS=(PROTOCOL=TCP)(HOST=primary-scan)(PORT=1521)))

(ADDRESS_LIST= (LOAD_BALANCE=on) (ADDRESS=(PROTOCOL=TCP)(HOST=secondary-scan)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=gold-cloud)))

Automa/cRetries

15

NorelianceonDNS

ALWAYSuseaservicethatisNOTDB/PDBname

ConfigureinOnePlaceLDAPorTNSnames


UseFANtoEliminateTCPTimeouts,Drainonplanned

16

OnbyDefault,Auto-Configured

Supports11.2,12.1,andlaterdatabases

12cR2FANAll Applications Driversandpoolsbreak-out&drain


12cFANOnbyDefaultandAuto-Configured

AllOracleuseONSJDBCUniversalConnecUonPool

JDBCThinDriver(12.2)

OCI/OCCIdriver

ODP.NETUnmanagedProvider(OCI)

ODP.NETManagedProvider(C#)

OCISessionPool

WebLogicAcUveGridLink

Tuxedo

Listeners

17

Auto-Configured

DESCRIPTION =

(CONNECT_TIMEOUT=90) (RETRY_COUNT=20)(RETRY_DELAY=3) (TRANSPORT_CONNECT_TIMEOUT=3)

(ADDRESS_LIST =

(LOAD_BALANCE=on) ( ADDRESS = (PROTOCOL = TCP) (HOST=primary-scan) (PORT=1521)))

(ADDRESS_LIST =

(LOAD_BALANCE=on)

( ADDRESS = (PROTOCOL = TCP)

(HOST=second-scan)(PORT=1521)))

(CONNECT_DATA=(SERVICE_NAME=gold)))

ONSNodeSet1

ONSNodeSet2


AlignyourTimeouts

FASTSTARTFAILOVER

(FSFO)

RACPrimary RACStandby

Applica/onTimeout(SLA)

MISSCOUNTorEXASpecial

FASTSTARTMTTRTARGET(FSMT)

ACREPLAYTIMEOUT

Application Timeout Ø FSFO + Crash Recovery Ø 2 x MISSCOUNT + FSMT

CrashRecovery(FSMT+Open)

ACREPLAYTIMEOUT


EasyStepsforApplica/onCon/nuityUnplanned outages should be hidden from applications

3

19


§  Replaysin-flightworkonrecoverableerrors

§  Maskshardware,so\ware,network,storageerrorsandUmeouts

§  12.1JDBC-Thin,UCP,WebLogicServer,3rdPartyJavaapplicaUonservers

§  OCI,ODP.NETunmanaged,JDBCThinonXA,Tuxedo,SQL*Plus

§  RAC,RACOne,&AcUveDataGuard

In-flightworkcon/nues

ApplicaUonConUnuity

20

12.2


ApplicaUonConUnuityDemonstraUon

21


UndertheCovers

1–NormalOpera/on

• Clientmarksdatabaserequests

• Serverdecideswhichcallscan&cannotbereplayed

• Directed,clientholdsoriginalcalls,theirinputs,andvalidaUondata

2–OutagePhase1:Reconnect

• Checksreplayisenabled

• VerifiesUmeliness

• CreatesanewconnecUon

• Checkstargetdatabaseisvalidforreplay

• UsesTransacUonGuardtoguaranteelastoutcome

3–OutagePhase2:Replay

• Replayscapturedcalls

• Ensuresresultsreturnedtoappmatchoriginal

• Onsuccess,returnscontroltotheapplicaUon

22


EasyStepsforApplicaUonConUnuityCheck Whattodo

IdenUfyRequests UseOraclepoolsandreturntheconnecUons,oraddboundaries

NodeprecatedJava UseORAChktoknownodeprecatedJava(Javaonly)

SetIniUalState Donothingformostapps,FAILOVER_RESTORE=LEVEL1formostothers

GrantKeep Grantkeepingmutablevalues,e.g.seq.nextval,sysdate,sysUmestamp

Skiparequest UseadifferentconnecUonordisabletoskipreplayingarequest

ReportProtecUon UseORAChktoreportyourcoverage


RuntheACAssessmentsHoweffecUveisApplicaUonConUnuityforyourapplicaUon?WhereApplicaUonConUnuityisnotineffect-whatstepsneedtobetaken?

No Steps

1AnalyzeandReportCoverage

2ReportusageofdeprecatedJavaClasses

AvailableinORAchk

Assessmenttoolinput

output

ApplicaUontraces

userOutput

orachk

read

hjps://blogs.oracle.com/WebLogicServer/entry/using_orachk_for_coverage_analysis 24


• Applications that borrow a connection and do not return until completed

• Purges committed transactions as its goes • Session states reset and replay through outside your TX • Rules apply

12cR2Applica/onCon/nuityforLongRunning

25

26 © NEC Corporation 2015

History of NEC/Oracle alliance

2005 STA (Strategic Technology Alliance) started

NEC and Oracle OEM contract started(first in Japan)1987

2000 Development alliance for mission critical systems

NEC and BEA alliance started1997

2012

2013

2015

2006 NEC’s RAC 10gR2 fast failover best practice

NEC high available Linux DB platform

Zero error planned DB maintenance and unplanned DB outage solution

Zero error solution enhancement for cloud

The NEC and Oracle alliance is continuous over a quarter of a century

2008 NEC’s RAC 11gR1 fast failover best practice Won Global Partner Award:

Database 2015,2016


DB Instance Down

Application wait FSMT (max)

Reconnect

FAN

: request and response

…RAC Node#1

WebLogic

RAC Node#2

PDB servicesPDBs

PDB services PDBs

DB Reconfig & Recovery

instancedown

No Errors!!!

Confirmed no errors to all tenantʼs clients

Application Continuity


Public Network Down

Application wait 1s

FAN


…RAC Node#1

WebLogic

RAC Node#2

PDB servicesPDBs

PDB services PDBs

PublicNetwork

down

No Errors!!!


Reconnect



DB Node Down

Application wait Misscount+FSMT (max)

FAN


…RAC Node#1

WebLogic

RAC Node#2

PDB servicesPDBs

PDB services PDBs

GRID + DB Reconfig & Recovery

Nodedown

No Errors!!!


Reconnect



REDO sync

PDB Online Relocate

Other PDBs

Other PDBs

Migration PDB

requestandresponse

CDB#1

WebLogic

…

CDB#2

(1) Relocate

Online Copy

(2) Open &Start Service


Application Continuity enables no error replay


ApplicaUonConUnuityPerformanceWebLogicServerAc/veGridLinkandRealApplica/onClusters

APserverCPU DBserevrCPU

CPUpertransacUon

APservermemory

MemorypertransacUon

0200400600

select&update

Throughput(tx/s)

0

10

20

select&update

ResponseUme(ms)

■ ACOFF■ ACON

MedRecApplica/on

31


LessonsLearned•  AlignUmeoutstoallowfailoverandreplay

•  ReturnconnecUonstopoolbetweenrequests.

•  UsetherecommendedTNStoauto-configureFANandRetry

•  Usemutablevalues.ThinkofmutablesintermsofdelayedexecuUon.

•  IftheapplicaUonsetsvaluesoutsidetheapplicaUon–useFAILOVER_RESTORE

•  IftesUngandusingV$instanceetc,selectfromthedriveritself,notinrequests

32

©2014 Epsilon. Private & Confidential

Real World Experience Improving Application Continuity at Epsilon

Gairik Chakraborty

33

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

34

•  Snapshot of Epsilon •  Production Implementation

Agenda

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

35

Snapshot of Epsilon

Proprietary and Confidential

35

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

36

•  Epsilon is all-encompassing global marketing company, we are global leader in turning data-driven marketing into lasting relationships.

•  More than 7000 associates and 70 offices worldwide

•  Largest permission-based e-mailer in the world, delivering over 47 billion emails annually

•  World’s leading source of data with information covering over 250 million consumers and 273M devices

•  More than 2,000 global clients, including 26 of the Fortune 100

!  9 out of 10 Top Banks

!  8 out of 10 Top Retailers

!  The Top 10 Pharmaceutical Companies

!  The Top 10 Automotive Companies

Epsilon at a Glance

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

37

Marke/ngDataHelpsBrandtounderstandcustomersandconnectwiththemInsightsandStrategyUnderstandwhatcustomersandprospectsaredoingandthinking.HowandwhentheyengagewithbrandsMarke/ngTechnologyTechnologyplavormsenablesconnecUonbetweencustomerandBrandsCrea/veServicesConnectscustomertobrandsatemoUonallevelthroughstrategicthinking,creaUvedigitaldesign,artdirecUon

AgilityHarmonyNextgeneraUondigitalmessagingplavormdevelopedbymarketersforeffecUvemulUchannelcampaignsincludinge-mail,mobileandsocialmedia

Epsilon at a Glance

MediareachAlwayshaverightdigitalorofflinepresencetodelivertargetedmessagestopotenUalcustomers

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

38

•  Senior Director, Database Technology team

•  High availability and Engineered Systems implementation for multiple large fortune 100 clients

•  Working on Big Data and Cloud deployments for Marketing technology platforms

•  Over 15 years of experience working on Oracle technology platform

•  OCP and OCE ( DB 12c, RAC and Exadata )

About Me

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

39

Improving Application Continuity at Epsilon

Proprietary and Confidential

39

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

40

•  Global customer with extreme performance and availability requirements

•  Zero downtime requirement for entire application stack

•  System needs to be fault tolerant including entire site failure

•  Maintenance needs to be performed while system is online

•  Real-time monitoring and reporting of system performance and health

High Level Business Requirements

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

41

Previous State

•  Database version 11gR2 RAC 11.2.0.4

•  Application was using WLS 12.1.3

•  Active Data Guard with reporting workload running from DR site

•  Application servers sitting idle on DR site

Previous State and Challenges

Challenges

•  Network maintenance in a data center requires application downtime

•  Significant investment for DR site, most of the resources stays idle

•  No protection for unplanned outages including node failures

•  Many manual steps required to switch over application to DR site and process involves downtime

•  Customer wanted zero downtime solution

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

42

Actions

•  Database upgraded to 12c – 12.1.0.2

•  Application is now using WLS 12.2.1

•  Database hardware platform using Exadata X5-2

•  Active / Active Replication between the sites using GoldenGate – version 12.2

Current State and Resolutions

Outcome

•  Connection pool drains quickly after receiving FAN events

•  No longer application server restart required for planned maintenance or unplanned outage of Oracle stack

•  Application Continuity replays transactions during unplanned outage – which would have failed otherwise . No impact to user experience

•  WLS coherence federated caching feature helps to replicate cached data across multiple data center and keeps application server in sync

•  GoldenGate replication keeps databases in sync at both data centers

•  Moved to a supported release good till 2018

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

43

Technology Stack

•  Exadata X5-2 Primary and DR site •  Application servers use :

•  WebLogic Server 12.2.1 with Active Gridlink •  Federated Coherence caching

•  Oracle Database 12c (12.1.0.2) •  Real Application Clusters (RAC) •  Active Data Guard •  Fast Application Notification (FAN) •  Application Continuity (AC) •  Transaction Guard (TG)

•  Oracle GoldenGate ( 12.2 ) for active - active replication

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

44

Web Server cluster WLS 12.2.1

Application Server Cluster WLS 12.2.1

OLTP Batch

OLTP Service

Batch Service

RRAC Node 1 12.1.0.2


Site 1

Site 1 DB



OLTP Batch

Site 2

Normal Operation : Application Service Placement

Federated Caching

OLTP Service

Batch Service



Site 2 DB

Federated Caching

Golden Gate Replication

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

45



OLTP Batch

OLTP Service

Batch Service

RAC Node 1 12.1.0.2


Site 1

Site 1 DB



OLTP Batch

Site 2

Scheduled Maintenance: Application Service Placement

Federated Caching

OLTP Service

Batch Service



Site 2 DB

Federated Caching


Fast Application Notification

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

46



OLTP Batch

OLTP Service

Batch Service

RAC Node 1 12.1.0.2


Site 1

Site 1 DB



OLTP Batch

Site 2

Unplanned Outage: Application Service Placement

Federated Caching

OLTP Service

Batch Service



Site 2 DB

Federated Caching


Fast Application Notification Transaction Guard Application Continuity

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

47

Application Continuity Assessment 1. Clean up Concrete Classes

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

48

Concrete class check - results

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

49

Action Item

•  Oracle recommends to replace concrete classes with new interfaces

•  Refer My Oracle support note 1364193.1 for more details

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

50

Application Continuity Assessment 2. Coverage Analysis - results

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

51

Database side configuration

1.  Enables Transaction Guard 2.  Enables application Continuity

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

52

Special Considerations

•  Need to think about sequences , sysdate , systimestamp , SYS_GUID usage in application. •  If original values are not preserved , replay will be rejected GRANT [KEEP DATE TIME | KEEP SYSGUID].. [to USER] GRANT KEEP SEQUENCE.. [to USER] on [sequence object]; ALTER SEQUENCE.. [sequence object] [KEEP|NOKEEP]; •  Kill session while Application Continuity is enabled at service level should use noreplay clause: alter system kill session 'sid, serial#, @inst' noreplay; alter system disconnect session 'sid, serial#, @inst' noreplay;

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

53

Oracle JDBC Replay Driver

•  At WebLogic Server choose Application Continuity enabled JDBC data source

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

54

Application Continuity tracing

•  RDBMS

alter system set event='10602 trace name context forever, level 28:trace[progint_appcont_rdbms]:10702 trace name context forever, level 16' scope = spfile ;

•  Web Logic Server

-Dweblogic.debug.DebugJDBCReplay=true •  JDBC driver

-Djava.util.logging.config.file=configfile -Doracle.jdbc.Trace=true Reference : https://docs.oracle.com/middleware/1221/wls/JDBCA/ds_oracledriver.htm#JDBCA787

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

55

Lessons Learned

•  Use JDBC statement cache instead of WebLogic statement cache

https://docs.oracle.com/middleware/1221/wls/JDBCA/ds_oracledriver.htm#JDBCA608 •  For Application Continuity assessments, refer following : https://blogs.oracle.com/WebLogicServer/entry/using_orachk_to_clean_up https://blogs.oracle.com/WebLogicServer/entry/using_orachk_for_coverage_analysis

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

56

Business Benefits

•  Scheduled maintenance ( Database and OS patching ) of Oracle technology stack can be done without disrupting business user experience. ( meet security compliance as well as uptime SLA )

•  Application restart is no longer required after planned maintenance or unplanned outages.

•  No application call failures seen in case of unplanned outage improves end user experience

•  Using Oracle technologies - Real Application Clusters on Exadata -Fast Application Notification Transaction Guard , Application Continuity, Golden Gate and Web logic Server coherence caching, Epsilon is able to deploy zero downtime solution across multiple sites.

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

57

Next Steps

•  Use Application Continuity for ODP.NET to hide unplanned outages without any application code change

•  Implementation of zero downtime solution for long running batch workload

©2014 E

psilon Data M

anagement, LLC

. Private &

Confidential

58

Q&A?


Transac/onGuardCodeyourown–hideunplannedoutages

4

59


§  AllowsapplicaUonstodealwithfailuresandUmeoutscorrectly

§  WithoutTransacUonGuard,retryingcancauselogicalcorrupUon

§  ApplicaUonConUnuityusesTransacUonGuard

§  JDBC-thin,OCI/OCCI,ODP.NETunmanaged

§  ODP.NETmanaged,XAapplica/ons

GuaranteedCommitOutcome

TransacUonGuardFirstRDBMStopreserveCOMMITOutcome

12.2

60


authenUcate…….

…….

COMMIT;

<getanewsession>Forcecommitoutcome

COMMITTED?COMPLETED?

assignLTXIDstarttransacUon

Session

Oracle12cDrivers Oracle12cDatabase(s)

Time

ErrororUmeoutCOMMIT

SQL,PL/SQL,RPC

LTXID

GET_LTXID_OUTCOME

NewSessionSameDBImage

Preserve&ReturnCOMMITOUTCOME

HowTransacUonGuardWorks

61


TransacUonCoverageInclusions

Local

CommitonSuccess(auto-commit)

DistributedandRemote

DDL,DCL,parallelDDL

PL/SQLwithembeddedCOMMIT

PL/SQLwithCOMMITaslastcall

Read-only(allowedfor)

XAforone-phaseop/miza/onsandpromotableXA

Exclusions

AcUveDataGuardwithdatabaselinksusedtocommitatprimary

62


TransacUonGuard–KeyTakeaway

FirstRDBMStopreservecommitoutcome

• UsersshouldnotseemisleadingerrorswhenatransacUonreallydidcommit.

• Oncethecommitoutcomeisreturned,theresultneverchanges.

• SafetoreturnsuccessorresubmitforapplicaUonsandmid-Uers

63


NEC is ready to provide Highly Available and Maintainable Cloud platform with Application Continuity, Real Application Clusters, Oracle Multitenant, Oracle Security options and WebLogic Server Active GridLink on NEC HW. I believe this platform brings phenomenal value to our cloud customers because it hides unplanned outages and scheduled maintenance from application users.

Yuki Moriyama Deputy General Manager, NEC Corporation

OnlineDatabaseMaintenancewithNoImpacttoApplica/onsCON6757Wednesday,4:15pm,104MosconeSouth

64


NEC at Oracle OpenWorld 2016▌ NEC Sessionl Oracle Cloud AppDev for Enterprise Java Applications [CON7098]

•  Wednesday, Sep 21, 4:15 p.m. - 5:00 p.m. | Marriott Marquis̶Salon 4/5/6

NEC Booth #1601, Moscone South

▌ Exhibitionl Cutting edge technologies in Big

Data, Cloud, Security, SDN area

NEC


SafeHarborStatementTheprecedingisintendedtooutlineourgeneralproductdirecUon.ItisintendedforinformaUonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncUonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andUmingofanyfeaturesorfuncUonalitydescribedforOracle’sproductsremainsatthesolediscreUonofOracle.

66


Documents

CON6757 - Oracle · 36 • Epsilon is all-encompassing global marketing company, we are global leader in turning data-driven marketing into lasting relationships