SAS Viya : A Journey in Statistics Canada’s Universe

Karine DésiletsTeam Leader and Technical AdvisorSystem Engineering DivisionStatistics Canada

November 20th, 2018

The Importance of Data

“The world’s most valuable resource is no longer oil, but DATA” The Economist, May 2017

Source : Image taken and adapted from another Statistics Canada presentation

SAS enable organizations to thrive in the analytics economy

ANALYTICAL ECONOMY

Data+

Analytics+

Collaboration

CanapésStatistics Canada Analytical Economy

First CourseViya 3.3 Architecture and Clients

Loading Data in SAS Viya

Programming Languages and Snippets

Main CourseDeployment of SAS solutions – Macro Catalogs and Source Control in CAS (Informatics)

In-memory and multi-threaded algorithms (Mathematicians)Classifications and Proc Cardinality (Analytics)

DessertPros and Cons of SAS Viya in CAS and Statistics Canada

R&D – Future as extra

Conclude with improvements

Menu

Statistics Canada Analytical EconomyC

ompu

ter S

cien

ce • Architect• Programmers• System Analysts• BI Analysts• Platform

Administrator• Algorithmic

Specialists• Databases Experts• User Experience• Researchers • Security

Anal

ytic

s

• Economists• Data Scientists• Geographers• Statisticians• Sociologists• Industry Analysts• Environmental

Analysts• Criminology• Researchers

Met

hodo

logi

sts • Mathematicians

• Statisticians• Researchers

• Includes:• Statistical Methods• Spatial and Temporal

Estimation• Non-Sampling Variance • Outlier Detection• Benchmarking• Interpolation• Calendarization• Longitudinal Data Analysis• Time Series

What is Viya ? Which part is related to Statistics Canada?

SAS Viya

SAS Studio 4.x Series

SAS Cloud Analytic Services (CAS)

SAS Visual Analytics

SAS Visual Data Mining and Machine Learning

SAS Econometrics

SAS Visual Forecasting

SAS Optimization

See more details with : An Introduction to SAS Viya : What is it and what it isn’t! byFrank Scott Analytic Solutions Architect, Canadian Public Sector, SAS Institute (Canada)

SA

S V

IYA

3.3

-Fu

ll D

eplo

ymen

t

Source : support.sas.com (https://documentation.sas.com)

As an employee – What do you need to know ?

Source : support.sas.com (https://documentation.sas.com)

SAS Viya 3.3 is compliant to SAS programming run-time environment (SPRE)

(Deployment)

The parallelisation occurs at the procedures level.

Viya Clients – What do you need to know ?

What is a server without a client?

Web Client

Mobile Client

SAS Home on Viya

SAS EG

SAS DI

Clients

SAS® Clients

9

CAS-L CAS ProcCAS Actions

Pyth

on

Rest Interface using JSON

Lua

Base SAS Compliant

Java

R

That’s Great! All those languages in a SAS program ?

SAS Viyaand

ProgrammingLanguages

The Reality – What can you do ?

SAS Studio 4.x ,SAS EG & SAS DI 9.4M5• CAS actions, CAS-L Language

and new CAS Procedures• Data Step, DS2, FedSQL can run

in CAS Server (no mention of proc sql)

• Base SAS code minimally compliant to execute on SAS Workspace Server

Python 2.7.x & 3.4.x, Lua 5.2/5.3, R 3.1.x • CAS actions, CAS-L Language

and new CAS Procedures

• SAS provides the SAS Wrapper for Analytics Transfer (SWAT)

• Comparison of models built in these different languages in SAS Model Manager

Java 8REST API• CAS actions, CAS-L Language

and new CAS Procedures

• SAS Java Interface for Viya) –with a client JAR file

• With REST API, CAS Offers an embedded HTTPs server that can be accessed for running actions

Major Programming Differences

Data Steps• Process in CAS server• Process in a SAS session• Parallelize With Group By• Take advantage of the in-memory

infrastructure :SMP ModeMPP Mode

No Proc sort neededNo rows level access

Macros• Supported in SAS Viya• Process in SAS session• The macro language itself do

not run on CAS.• Macro with CAS procedures run

in-memory

User-Defined Formats• Stored in Catalogs and process

in a SAS Session

• Stored in a Format library on the CAS server. Format libraries can be associated to a CAS session

• PROC FMT2ITM

SAS Viya : What does it Look Like ?

SAS Viya : What does it Look Like ? (2)

SAS Viya : What does it look like ? (3)

Yes but Viya is great, it gives you access to a Snippets bank of SAS code

15

What is a Snippet ?

SAS Code snippets enable you to quickly insert SAS code into your program and customize it to meet your needs.

Want to use Viya ? What do you need ?

Options:

• SAS Data Explorer – Manage Data

• Programming Interface

• SAS DI / SAS EG

• Probably other tools

How will I load the Data in Viya ?

17

Wait … A More Complex Scenario

I am an IT Folks!I’ve got generalized SAS Solutions.

I want to Deploy SAS solutions and have it accessible to my clients…

What are the options!

IT Perspective – What is the SAS Actual Deployment Model in Relation with Viya?

SAS Macro Catalogs

Dependant of :• Operating System• SAS version Compliant

SAS Programs

I am a SAS Studio User!

19

I am a super Methodologist dealing with computer-intensive calculations and I want the results within seconds!

How will I use In-Memory Multi-Threaded Procedures efficiently in Viya ?

Steps to Follow:

1. Create a CAS connection2. Bind a libname the your current

Data and CAS session3. Load the Data in-memory4. Process the tremendous

Calculations5. Save your Results in Data steps or

report it !

Demo of SAS Code

21

We are a bunch of Analytics people dealing with Classifications and we would like to know if the data we got respect the classification.

How I could use Proc Cardinality efficiently in Viya ?

Immigrant Status

Hou

seH

old

Type

Total Income

Legal Marital Status

National Occupational Classification (NOC)

Inco

me

Sour

ce

North American Industry Classification System (NAICS)

North American Products Classification System (NAPCS)

Gender / Sex

Afte

r Tax

-Inco

me

Duration of Commute

First Official Language Spoken

Proc Cardinality – A Data Prep Procedure

Price Data from Jan98 to Dec02 – 3 Regions – 17 Products

Pros, Cons and Other Considerations

ProsIn-memory process data faster (RAM)

If not in-memory, process on Workspace server (ROM) – no need to modify vintage programs

SAS PROCs and language has been clean-up for in-memory processing. Ex: No more proc Sort

ConsThe In-memory infrastructure is expensive in comparison to Disks

All applications need to be tested with the new language and/or new infrastructure

Load data in-memory could have processing time degradation

Not all PROCs are compliant to CAS

Other ConsiderationsWith Viya – What would be the versioning of SAS in Cloud

Current STC network B bandwidth need to be adjusted – Some SAS Viya interface not loadable

Cloud Agnostic – GoC need a solution that can work with whichever the Cloud provider chosen

24

Hey ! I heard SAS is doing some R&D in order to haveSAS GRID MANAGER and SAS VIYA Combined Together

Come with me, we will look at that…

SAS GRID Manager and SAS Viya : a Perfect Pair

Statistic Canada SAS GRID allows multi threading at the Jobs Level on disk

SAS Viya allows threading at the procedure level with in-memory processing

Combine Both ? – WOW! What Super Powers you would have! Ken Gahagan : A perfect pair :https://www.sas.com/en_us/webinars/sas-grid-manager-sas-viya-a-perfect-pair.html

• SAS products centralized on a Hub with the Viya Platform• New possibilities with in-memory versus disk-based• CAS-L Language, CAS Procedures and CAS Actions

Source of Enhancements / Reflexions:

Compliance for the Future Vision• Statistics Canada is part of Government of Canada Plans and Roadmap Need to be compliant to GoC and Treasury Board of Secretariat, do have to

respect The Business Needs for Security (BNS) and IT Security Guidelines (ITSG).• Exploration of other components by experts in those fields

Conclusion

• SAS/EG Functionalities• SAS Studio visual access to folder /home/&userid

and writing (W) access to other directories

• Source Control • Password Management

27

SAS® 9.4 and SAS® Viya® 3.3 Programming Documentation - Orientationhttps://documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.3&docsetId=pgmmvaov&docsetTarget=pgmsasov.htm&locale=en

An Introduction to SAS® Viya® 3.3 Programminghttps://documentation.sas.com/api/docsets/pgmdiff/3.3/content/pgmdiff.pdf?locale=enSAS-Viya-3.3-Adminstration-Fundamentals-Course-Notes

SAS VIYA Free-Learning Introductory Web Videos:https://support.sas.com/edu/schedules.html?id=4936&ctry=CA#s1=1

https://documentation.sas.com/api/docsets/whatsdiff/3.2/content/whatsdiff.pdf?locale=en

DataStep in Sas Viya : Essential new Features, https://support.sas.com/resources/papers/proceedings17/SAS0118-2017.pdf

Getting Your SAS® 9 Code to Run Multi-Threaded in SAS® Viya® 3.3 https://blogs.sas.com/content/sgf/files/2018/01/TechnicalWhitePaper_SASViya_Jan2018_FINAL.pdf

I strongly recommend the reading of …

Statistics Canada : One hundred years and counting

THANK YOU!For more information please visit, www.statcan.gc.ca

#StatCan100

KARINE DÉSILETSTechnical Advisor and Team Leader

System Engineering DivisionStatistics Canada100 Tunney’s Pasture DrivewayOttawa, Ontario K1A 0T6karine.desilets@canada.ca613-862-4220