Creating custom visuals withPower BI Visuals CLI Denys Chamberland Senior Web Programmer-Analyst @MawashiKid mawashikid@gmail.com

MSDEVMTL – Data Platform ©2017

Denys Chamberland • Senior Web Programmer-Analyst for over 20 years• Significant collaboration in the architecture, coding and analysis design of several Web applications projects.• Microsoft Community Contributor on constant lookup of latest Microsoft technologies.• Montreal Devops Hackathon 2016 – second place and mention of honor.• Multimedia background: 10 years of experience in digital video production. • Expertise in photography and computer graphic design [hobby] …

@MawashiKid

MawashiKid@gmail.com

Agenda Introduction Custom Visual Gallery – PBIVIZ file format description and usage. Roadmap: Brief review of previous Power BI Visuals deprecated versions Basic concepts:

IVisual LifeCycle : contructor, update, destroy – object enumeration Capabilities : DataRoles & DataMapping, DataView, Object Formatting

Power BI Visuals ToolChain requirements installation Node.js + Node Package Manager (npm) Visual Studio Code Power BI Visuals CLI Tool (pbiviz) - via Node Package Manager (npm) Typings (gestion de fichiers) – via Node Package Manager (npm) Local Self-Signed Certificate – via pbiviz

Power BI Visuals CLI project architecture Demos Questions

Power BI business analytics tools already allow us to analyze business intelligence data and convert them in rich visual components such as dashboards and interactive reports.

Introduction

Introduction

Power BI Visuals

Fully aware of the constant fast growing business intelligence analysis needs and in an initiative to reach a larger scope of user profiles, Microsoft has brought a very important new key element allowing developers to create customized visual components - even beyond traditional visual approaches - with the addition of the Power BI Visuals tool.

BI strategies are no longer focused soleley on conventionnal basic options, but are now increasingly oriented towards a multitude of new range of interesting and performing possibilities.

The 3 main key elements used in Power BI Visuals custom visuals development are: TypeScript Less D3.js

Introduction

Introduction – key element :TypeScripthttp://www.typescriptla

ng.org/

All visual source code (src / visual.ts - and all the modules and interfaces they implement) must be written in TypeScript, which is a JavaScript superset and supports more advanced functionality and early access to ES6/ES7 features

All .ts TypeScript file content must be stored in the src / directory and listed under the files section in tsconfig.json.

Introduction – key element: Lesshttp://http://lesscss.or

g/• By default, all style sheets content (style/visual.less) is created in .less format.

• Less is a CSS pre-compiler that supports some advanced features such as nesting, variables, mixins, conditions, loops,....

• If you do not want to use these less functions, you can simply write your styles in CSS format.

 style/ 

Introduction – key element: D3.js – Data Driven Documentshttps://github.com/d3/d3/wiki/Gallery

• The last key element (though not the least ...) is the use of JavaScript open source frameworks..• A lot of the custom visuals available online on the Custom Visuals Gallery site of the Power BI community

were indeed built using D3.js. This approach is an open source data visualization framework created by Mike Bostock - who was one of the main collaborators in data visualization for the New York Time until 2015. An entirely JavaScript approach mainly focused on DOM manipulation and vectors usage optimization.

• An impressive collection of really neat types of visuals is available on the gallery of D3.js. Most samples can be easily reoriented in a Power BI Visuals context, without having to constantly reinvent the wheel.

Custom Visuals Gallery• https://app.powerbi.com/visuals/

The Custom Visuals Gallery hosts a collection of custom visual components created by Microsoft as well as independent developers who are members of the Power BI Community

Sandboxing: Since April 2016, all custom visuals run in a sandboxed IFrame. This allows you to use particular versions of libraries and global styles, without the fear that you'll step on anyone’s toes..

N.B: Most of the visuals found on the Custom Visuals Gallery were previously built using the Power BI Visuals CORE [deprecated]. Some - but not all - have been ported to the newPower BI Visuals CLI.

Custom Visuals Gallery• https://app.powerbi.com/visuals/

Clicking on any of the icons will then display a window screen with the name of the visual, the name of the author, We can alsoa brief description and a graphic image of the selected custom visual.

We can also contact the author and gain further detailed support informations. Some will even provide a reference link to the initial project source code on Github.

These visual components are available both in in .pbiviz package format as well as full .pbix demo sample.

Custom Visuals Gallery – how to import .pbiviz package• 1 Select the following menu option

File>Import BI Custom Visual • 2- A Caution: Import Custom Visual warning

message advises users that the custom visuals are not provided by Microsoft…

• 3 – Locate the .pbiviz package file to import

• 4 – A confirmation message should then indicate that the visual has been imported successfully

• 5 – Finally, an icon identifying the newly imported custom visual is then added to the Visualizations panel.

Power BI Visuals Extensibity Developer Tool- Deprecated

Retrospect https://app.powerbi.com/devTools

A first [very embryonic online DevTool "playground" style version] of online development tool was previously created as a launch pad at a Microsoft contest in September 2015. The contest which ended in October 2015 invited any participants to submit their visual creations.

Power BI Visuals Extensibity Developer Tool- Deprecated

Retrospect

Upper left screen used to include your visual TypeScript source code. Lower left screen used to include your visual CSS Stylesheet Main right screen to display the rendered visual when clicking on Compile & Run

button.. Finally the .pbiviz package compilation of the visual was performed by clicking on

Export button. Some TypeScript features – as let and const – were not fully supported and the IDE did not really offer any possibilities to manage external client libraries.

Power BI Visuals Extensibity Developer Tool- Deprecated

Retrospect

• An online upload form was previously available with the former release deprecated version.

• Submitting .pbiviz package to the Microsoft Custom Visual Gallery portal is now done through email

[ref: https://app.powerbi.com/visuals/info#submit ]

Power BI Visuals CORE- DeprecatedRetrospect https://

github.com/Microsoft/PowerBI-visuals-core

Microsoft then decided to consolidate an impressive list of visual components into a Visual Studio 2015 Web application with the Power BI Visuals CORE version.

Although the application provided a respectable amount of useful information, the architecture of the project itself was rather a bit too heavy to manage, requiring considerable amount of time and patience dealing with the Node.js, Gulp configuration processes.

The application also did not really offer options for creating .pbiviz package "right out-of-the-box“…

Adding new visual components required cloning an existing directory, rebuilding the code, and rolling a Gulp task whenever you wanted to perform a simple .pbiviz package compilation.[ref: http://amolpandey.com/2016/08/02/powerbi-how-to-create-pbiviz-file-custom-visualization/]

Power BI Visuals CORE- DeprecatedRetrospect

A "playground" client application could be launched from Visual Studio 2015 (Ctrl + F5), allowing users to view all the code examples included in the Power BI Visuals CORE project - either in Web View mode.

Power BI Visuals CORE- DeprecatedRetrospect

Or in Mobile View mode. [Sorry folks, no Export .pbiviz button…]

Power BI Visuals CLI – PBIVIZ ToolRetrospect

• Creating a new pbiviz project: pbiviz new <NomdeProjet>• Updating an existing pbiviz project: pbiviz update• Compiling a project in pbiviz package: pbiviz package• Starting local server in order to debug project on Power BI Online: pbiviz start

After having carefully listened to several comments from developers in Power BI Community, Microsoft then decided it was time to modify its former previous deprecated approach and to rethink a more compact, more malleable and homogeneous IDE environment approach which would also offer a better management External client libraries.

Then in August 2016, Microsoft officially announced the release of the new version of Power BI Visuals CLI.

The new Power BI Visuals CLI development tool is entirely based on the pbiviz command line and works on all platforms with any npm compatible IDE of your choice.

The main pbiviz command options are:

Power BI Visuals CLI – PBIVIZ ToolRetrospectA simple CLI pbiviz new <NomdeProjet> command now allows you to generate a new Power BI Visuals CLI project template in a few seconds. Et voilà …

Basic concepts: IVisual LifeCycle

constructor(options:VisualConstructorOptions): void;

update?(options: VisualUpdateOptions): void;

destroy?(): void;

The IVisual interface is the main engine of Power BI Visuals which any visual component class [visual.ts] must implement. The IVisual interface lifecycle is performed in 3 steps:

The constructor method is called only once, when the visual is initialized for the first time during loading.

The update method is invoked whenever an update is performed on the visual component, such as modifying data source, changing parameters properties or during a resizing or repositionning carried by the user.

Finally, the destroy method is called when the visual is bout to be disposed. Here the visual should be null as well as all attached resources to avoid memory leaks.

Basic concepts: capabilities.json 1- dataRoles 2- objects

• The capabilities.json in new Power BI Visuals CLI architecture allows a better SOC [separation of control].

• Microsoft team soon realized that the formatting management of Power BI user interface did not really belong to the logic behind the visuals [visual.ts] source code.

• However it was deemed preferable to declare a list of objects options in capabilities.json configuration file that the Power BI environment could manage and support when loading the custom visual.

Basic concepts: capabilities.json – Data Roles & Mapping

Basic concepts: capabilities.json – DataViewWhat is a DataView?

DataView is a JavaScript object model for canonical representations of data

Initially, a DataView is not really a JSON format model, but rather a cross-referencing tree structure or object graph supporting multiple, single and canonical views of datas. This approach gives the visuals the freedom to choose their preferred model of canonical representions.

The DataView canonical structure are: metadata categorical table single matrix tree

Basic Concepts: capabilities.json – DataView

If you wish to see those datas ("metadata", "categorical", "matrix", "table"  ,. "single" et "tree") during live on-line testing in Power BI Online Developer, you simply have to click on DataView json format display

Capabilities – DataView- metadata

metadata:

The metadata provides information about the different columns, such as their type (categorical or scalar), the type of string that apply to its value, and static formatting options such as background color, legends layout position...

Capabilities – DataView- categorical

Capabilities – DataView- table

Capabilities – DataView- single

Capabilities – DataView- matrix

Installing Node.JS (free…) Version 5.0 recommended - 4.0+ minimum Installing Node Package Manage (npm)

  Installing Visual Studio Code (free…)

Visual Studio Code combines the simplicity of a source code editor with powerful developer tooling, like IntelliSense code completion and debugging…

Installing Typings (free…) Highly recommended to manage and install TypeScript definitions (typings.json)

Installing Power BI Visuals CLI tool (pbiviz) (free…) Installing via Node Package Manager (npm)

npm install –g powerbi-visuals-tools Installing Local self-signed certificate (free…)

pbiviz -–install-cert

Power BI Visuals CLI – ToolChain

• https://nodejs.org/en/download/

ToolChain - Node.js

• http://code.visualstudio.com/ ToolChain - Visual Studio Code

ToolChain - Typings - npm• https://www.npmjs.com/package/typings

# Install Typings CLI utility.npm install typings --global

# Search for definitions.typings search tape

# Find a definition by name.typings search --name react

# If you use the package as a module:# Install non-global typings (defaults to "npm" source, configurable through `defaultSource` in `.typingsrc`).typings install debug --save

# If you use the package through `<script>`, it is part of the environment, or# the module typings are not yet available, try searching and installing with `--global`:typings install dt~mocha --global --save

# If you need a specific commit from github.typings install d3=github:DefinitelyTyped/DefinitelyTyped/d3/d3.d.ts#1c05872e7811235f43780b8b596bfd26fe8e7760 --global --save# Search and install by version.typings info env~node --versionstypings install env~node@0.10 --global --save# Install typings from a particular source (use `<source>~<name>` or `--source <source>`).typings install env~atom --global --savetypings install bluebird --source npm --save

#Use `typings/index.d.ts` (in `tsconfig.json` or as a `///` reference).cat typings/index.d.ts

Typings is the simple way to manage and install TypeScript definitions. It uses typings.json which can resolve to the Typings Registry, Github, NPM, Bower, HTTP and local files. Packages can use type definitions from various sources and different versions, knowing they will never conflict for users

ToolChain - Power BI Visuals CLI Tool (PBIVIZ)

npm install –g powerbi-visuals-tools

pbiviz

In order to install the Power BI Visuals CLI (PBIVIZ) Tool, you can simply execute the following npm command from a Command Prompt(Admin) console

Once the installation process is completed, you can confirm that all operations have been performed successfully by executing the following pbiviz command – [without parameters]:

ToolChain - Power BI Visuals CLI Tool (PBIVIZ) A window displaying a list description of all available pbiviz command parameters options should then appear on screen:

ToolChain - Local self-signed certificate

Visual components viewing and testing can be performed live on Power BI Online site: https://app.powerbi.com

You must first select the Settings menu option.

ToolChain- Local self-signed certificate

Then under the General>Developer tab, simply make sure to check the “Enable Developer visual for testing” checkbox You must then ensure that your local server is started and

running by executing the following pbviz command from the root

directory of your project. [Ex: C:\PBIVisuals\barChart] pbiviz start

ToolChain- Local self-signed certificate

This indicates that Power BI Online is unable to detect a secure https server on your workstation.

To overcome this problem, you must first install an SSL certificateusing the following pbiviz command

Once the SSL Certificate is installed, this will allow your visual components to be loaded in your local broweser

pbiviz -–install-cert

Oooooops!!!... “Can’t contact visual server.

Please make sure the visual server is running and configured correctly”

ToolChain - Local self-signed certificate

Visual Studio Code – PBIVIZ CLI template project architecture

api/ This directory holds informations on the Power BI Visuals CLI api versionassets/ This directory is Used to store visual assets (icon, screenshots).

dist/ When you execute the pbiviz package command, a pbiviz file package is automatically generated in the dist/ folder. src/ This directory stores all TypeScript and JavaScript source code. style/ This directory stores the main visual.less which is is defined by default. You can import andr assign other CSS stylesheets as follows: @import (less) "style/external.css";.gitignore Manifest which tells git to ignore files that shouldn’t be tracked in the directory.

capabilities.json This configuration file is used to define all your visual capabilities [dataRole, dataViewMappings, object]

package.json This file is used to manage node_modules content through npm commands.

pbiviz.json Main configuration file of your project.

tsconfig.json TypeScript compiler settings.

typings.json This file is optional though highly recommended to manage TypeScript file definition

Demos…

Demo – Ferrari svg code [the power of vectors…]

Demo – Slicer – thermometer [PBI CLI upgraded version]

Demo – d3.js circlepacking [avoiding recreating whole DOM on update() ]

Questions?

Power BI Visuals Build Your Custom Visuals in Power BI (Step-by-Step) -Tsuyoshi Matsuzaki:

https://blogs.msdn.microsoft.com/tsmatsuz/2016/09/27/power-bi-custom-visuals-programming/ Create a custom visual with the custom visual developer tools

https://powerbi.microsoft.com/en-us/documentation/powerbi-desktop-tips-and-tricks-for-creating-reports/ How to Create a Power BI Custom Visual Based on an Existing d3 Visual – Jon Gallant

https://powerbi.microsoft.com/en-us/documentation/powerbi-desktop-tips-and-tricks-for-creating-reports/

Forums Community Power BI– (section Developer) https://community.powerbi.com/t5/Developer/bd-p/Developer

TypeScript: TypeScript Samples:

https://www.typescriptlang.org/samples/ TypeScript Deep Dive:

https://github.com/basarat/typescript-bookD3.js:

D3.js Gallery – Mike Bostock: https://github.com/d3/d3/wiki/Gallery

Collection D3.js Gallery (2490 examples!) – Christophe Viau :http://christopheviau.com/d3list/

D3 Tips and Tricks v3.x – Malcolm Maclean:https://leanpub.com/D3-Tips-and-Tricks

Useful links… [Thank you for your participation ;]