The lazy administrator, how to make your life easier by using tdi to automate your work - soccnx

The lazy administrator

How to make your life easier by using TDI to automate your work

About us…

Senior System Architect Senior Consultant

kbild.ch twitter.com/kbild

linkedin.com/in/kbild

wannes.rams.be twitter.com/wannesrams

linkedin.com/in/wannesrams

Agenda •  Introduc?on to TDI (a.k.a SDI)

– What is TDI – How to use it with Domino – How to use it with Connec?ons

•  Examples, examples, examples – Create a Wiki page with users of your Domino address book

– Export users last logon date per applica?on – Maintain Community membership through a Domino applica?on

Goal

Giving you a basic understanding how you can use

Tivoli Directory Integrator to reuse data which resides

in IBM Connec?ons or IBM Domino.

Who are you?

What is Tivoli Directory Integrator (TDI 7.1.1) aka Security Directory Integrator (SDI 7.2)

Input (Feed)

Assembly Line (AL)

Output

Func6ons Flow Components

Scripts A<ribute Maps

What is Tivoli Directory Integrator (TDI) aka Security Directory Integrator (SDI)

Modes: •  AddOnly (A) •  CallReply (C) •  Delete (D) •  Delta (Δ)

•  Iterator (I) •  Lookup (L) •  Update (U) •  Server (S)

What is Tivoli Directory Integrator (TDI) aka Security Directory Integrator (SDI)

Available Connectors (7.1.1, more than 60): •  Ac?ve Directory Change Detec?on Connector •  AssemblyLine Connector •  Axis Easy Web Service Server Connector •  Axis2 Web Service Server Connector •  CCMDB Connector •  Command line Connector •  Database Connector •  Deployed Assets Connector •  Direct TCP /URL scrip?ng •  custom •  Domino AdminP Connector •  Domino Change Detec?on Connector •  Domino Users Connector •  DSMLv2 SOAP Connector •  DSMLv2 SOAP Server Connector •  EIF Connector •  File Connector •  File Management Connector •  Form Entry Connector •  FTP Client Connector •  Generic Log Adapter Connector •  Old HTTP Client Connector •  HTTP Client Connector •  Old HTTP Server Connector •  HTTP Server Connector •  IBM MQ Connector •  IBM Directory Server Changelog Connector •  IdML CI and Rela?onship Connector •  IT Registry CI and Rela?onship Connector •  ITIM Agent Connector •  TIM DSMLv2 Connector •  JDBC Connector •  JMS Connector •  JMS Password Store Connector

•  JMX Connector •  JNDI Connector •  LDAP Connector •  LDAP Group Members Connector •  LDAP Server Connector •  Log Connector •  Lotus Notes Connector •  Mailbox Connector •  Memory Queue Connector •  Memory Stream Connector •  Proper?es Connector •  RAC Connector •  RDBMS Change Detec?on Connector •  SAP ABAP Applica?on Server Business Object Repository Connector •  SAP ABAP Applica?on Server User Registry Connector •  Script Connector •  Server No?fica?ons Connector •  Simple Tpae IF Connector •  SNMP Connector •  SNMP Server Connector •  Sun Directory Change Detec?on Connector •  System Queue Connector •  System Store Connector •  TADDM Change Detec?on Connector •  TADDM Connector •  TCP Connector •  TCP Server Connector •  Tivoli Access Manager (TAM) Connector •  Timer Connector •  Tpae IF Change Detec?on Connector •  Tpae IF Connector •  URL Connector •  Web Service Receiver Server Connector •  Windows Users and Groups Connector •  z/OS LDAP Changelog Connector

How to use TDI with Domino Available Connectors for Notes/Domino: •  Domino Change Detec6on Connector (Mode: I):

Enables TDI to detect when changes have occurred to a nsf database maintained on a Domino server and reports changed Domino documents.

•  Domino Users Connector (Mode: ADILU): Provides access to Lotus Domino user accounts and the means for managing them.

•  Lotus Notes Connector (Mode: ADILU): Works directly with any type of Notes Documents in any .nsf database.

•  Domino AdminP Connector (Mode: AI): The Domino AdminP Connector is a special version of the Lotus Notes Connector, the database parameter is always set to admin4.nsf. It has the capability to sign fields while adding a document and you can create AdminP request.

Or use non Domino specific: LDAP Connector (ADILUΔ) / HTTP Client Connector (AILC)

How to use TDI with Domino

Supported session types by Connector -‐> IIOP session gives you the highest flexibility

Supported Sessions > Connectors V

Local Client Session Local Server Session IIOP session

Domino Change Detec?on Connector

Yes No Yes

Domino Users Connector

Yes Yes Yes

Lotus Notes Connector

Yes Yes Yes

Domino AdminP Connector

No

Yes Yes

How to use TDI with Domino

If you are using IIOP sessions, perform the following: •  Ensure the Notes.jar file does not exist in the TDI_install_dir/jars folder and any of its subfolders.

•  Copy Domino_data/domino/java/NCSO.jar to TDI_install_dir/jars/3rdparty/IBM or to the folder specified by the com.ibm.di.loader.userjars property in global.proper?es (or solu?on.proper?es).

How to use TDI with Connec?ons Pre-‐packaged scripts with IBM Connec6ons: “Official” way to go if you want to change which users are imported or want to change/add/get profile data. Included scripts: collect_dns, delete_or_inac?vate_employees, dump_photos_to_files, dump_pronounce_to_files, fill_country/department/emp_type/organiza?on/workloc, load_photos_from_files, load_pronounce_from_files, mark_managers, populate_from_dn_file, sync_all_dns

Needs setup, has to be imported into TDI solu?on directory and will add two addi?onal connectors (Profile/Photo) as well.

IBM Connec6ons API: Gives you access to almost every func?on that you can access and use through the IBM Connec?ons user interface. You can use standard TDI connectors (i.e. HTTP Client connector). Be aware that the API documenta?on is not very good (to say it nicely).

How to use TDI with Connec?ons IBM Social Business Toolkit: TDI is java based and therefore you can use the IBM SBT SDK to create your own script connectors. You have to import some parts of the SDK into your TDI environment. You definitely should have a developer background. -‐> hmp://de.slideshare.net/AndreasArtner/ac?vity-‐stream-‐how-‐to-‐feed-‐the-‐beast Direct Database access: Connec?ons stores almost everything inside the RDBMS but there is no public DB schema info from IBM. This is not a supported way to change data inside Connec?ons (although some Partner solu?ons directly manipulate data in the database and their solu?ons are IBM supported). But you can use it to get data from Connec?ons.

Create a Wiki page with users of your Domino address book -‐ Example

Create a Wiki page with users of your Domino address book -‐ Example

Wiki page – How to 1.  Get all Domino users in names.nsf:

Just use Domino Users Connector in iterator mode, easy.

Best prac?ce: Always use property files for your parameters, it will save you a lot of ?me if you want to use the AL with different servers, environments!

Wiki page – How to 2.  Create the Wiki page Atom document

(AL create_Wiki_Entry_Atom): •  Find out how the Atom document has to be build

(hmp://www-‐10.lotus.com/ldd/appdevwiki.nsf/dx/Wiki_page_content_ic50) or try the SBT playground hmps://greenhouse.lotus.com/sbt/SBTPlayground.nsf/Explorer.xsp#api=Social_Wikis_API_Working_with_wiki_pages

•  Should be easy but… Example on SBT playground (does not work)

•  Works if you change the content line to <content type="text/html"><![CDATA[<p>This is James's wiki page.</p>]]>

Wiki page – How to 2.  AL create_Wiki_Entry_Atom:

•  Define the HTML code for the page •  Use the Prolog for the first part •  Use the iterator to generate the list •  Use the Epilog for the closing

Wiki page – How to 2.  AL create_Wiki_Entry_Atom:

•  This is the final code, all on ONE line: <?xml version="1.0" encoding="UTF-‐8"?><entry xmlns="hmp://www.w3.org/2005/Atom"><content type="text/html"><![CDATA[<div><p dir="ltr"><strong style="color: rgb(67, 106, 173);font-‐size:large;">All data is from the Domino directory -‐ Example for ICON UK </strong> <img src="/images/graphics-‐star-‐wars-‐300566.gif" width="151" height="100"/></p><table border="1" cellpadding="5" cellspacing="0" dir="ltr" style="border-‐collapse: collapse; width: 800px;" width="246"><tbody><tr height="14"><td><strong>Name</strong></td><td><strong>Shortname</strong></td><td><strong>Title</strong></td><td><strong>Company</strong></td><td><strong>Number</strong></td><td><strong>Photo (Connec?ons photo!)</strong></td></tr><tr><td><span class="vcard"><a class="fn url" href="">Chris?an Guedemann</a><span class="email" style="display: none;">[email protected]</span></span></td><td><span class="vcard"><a class="fn url" href="">CGU</a><span class="email" style="display: none;">[email protected]</span></span></td><td>Senior System Architect</td><td>WebGate Consul?ng AG</td><td><a href="sip://+41008008008">+41008008008</a></td><td><div style="width: 150px;height: 150px;border-‐radius: 75px;-‐webkit-‐border-‐radius: 75px;-‐moz-‐border-‐radius: 75px;background: url(/profiles/[email protected]) no-‐repeat;"></div></td></tr><tr><td><span class="vcard"><a class="fn url" href="">Klaus Bild</a><span class="email" style="display: none;">[email protected]</span></span></td><td><span class="vcard"><a class="fn url" href="">KBI</a><span class="email" style="display: none;">[email protected]</span></span></td><td>Senior System Architect</td><td>WebGate Consul?ng AG</td><td><a href="sip://+41004004004">+41004004004</a></td><td><div style="width: 150px;height: 150px;border-‐radius: 75px;-‐webkit-‐border-‐radius: 75px;-‐moz-‐border-‐radius: 75px;background: url(/profiles/[email protected]) no-‐repeat;"></div></td></tr><tr><td><span class="vcard"><a class="fn url" href="">Christoph Stoemner</a><span class="email" style="display: none;">[email protected]</span></span></td><td><span class="vcard"><a class="fn url" href="">CST</a><span class="email" style="display: none;">[email protected]</span></span></td><td>Senior IT Consultant</td><td>Fritz and Macziol GmbH</td><td><a href="sip://+41003003003">+41003003003</a></td><td><div style="width: 150px;height: 150px;border-‐radius: 75px;-‐webkit-‐border-‐radius: 75px;-‐moz-‐border-‐radius: 75px;background: url(/profiles/[email protected]) no-‐repeat;"></div></td></tr><tr><td><span class="vcard"><a class="fn url" href="">Sharon Bellamy</a><span class="email" style="display: none;">[email protected]</span></span></td><td><span class="vcard"><a class="fn url" href="">SBE</a><span class="email" style="display: none;">[email protected]</span></span></td><td>IT Consultant</td><td>Cube So{ Consul?ng</td><td><a href="sip://+41003003003">+41003003003</a></td><td><div style="width: 150px;height: 150px;border-‐radius: 75px;-‐webkit-‐border-‐radius: 75px;-‐moz-‐border-‐radius: 75px;background: url(/profiles/[email protected]) no-‐repeat;"></div></td></tr><tr><td><span class="vcard"><a class="fn url" href="">Wannes Rams</a><span class="email" style="display: none;">[email protected]</span></span></td><td><span class="vcard"><a class="fn url" href="">WRA</a><span class="email" style="display: none;">[email protected]</span></span></td><td>Social Business Consultant</td><td>GFI</td><td><a href="sip://+41003003003">+41003003003</a></td><td><div style="width: 150px;height: 150px;border-‐radius: 75px;-‐webkit-‐border-‐radius: 75px;-‐moz-‐border-‐radius: 75px;background: url(/profiles/[email protected]) no-‐repeat;"></div></td></tr></tbody></table></div> ]]></content><category scheme="tag:ibm.com,2006:td/type" term="page" label="page" /></entry>

Wiki page – How to 3.  Send the Wiki page Atom document to the Wikis API (HTTP

client connector): •  This is good documented (no joke)

hmp://www-‐10.lotus.com/ldd/appdevwiki.nsf/dx/Upda?ng_a_wiki_page_ic50

Wiki page – How to

This user needs editor rights on the Wiki

Wiki page – SSL requests •  Most Connec?ons environments force traffic over SSL •  If you get following error if you call the Connec?ons API

through SSL you have to import the Connec?ons server cer?ficate into TDI_install_dir/jserverapi/testadmin.jks (pw: administrator)

Wiki page – How to •  Final step is to create an AL with combines the

create_Wiki_Entry_Atom AL and the HTTP client connector

Export users last logon date per applica?on -‐ Example

Export users last logon date per applica?on -‐ Example

Export users last logon date – How to

•  We will export the last logon date for all users •  For all applica?ons •  Export to Domino •  Export to CSV •  This runs scheduled weekly as a repor?ng to our deployment

team


The workflow is as follows: 1.  Iterate through all entries in the PeopleDB and fetch uid and

full name 2.  Connect to applica?on table that contains profile 3.  Fetch user key 4.  Connect to Applica?on table that contains last logon date 5.  Repeat for all applica?ons 6.  Write to Domino 7.  Write to csv


•  Create a new assemble line and add a Database Connector. Make it an iterator and connect it to your Profiles database Employee table


•  I will show you for 1 database and then give you the mapping table for the other databases

•  Connect to the Files database, USER_TO_LOGIN table


•  Use the uid_lower as your key to find the relevant user key



•  Now connect to the Files database USER table to get the last logon date of this user using the USER_ID fetched in the last step as a link


•  Repeat these steps for all applica?ons, except Blogs. The Blogs database table ROLLERUSER contains uid and last logon date. On top of that it is the only table that uses the uid as is and not converted to lowercase.(thank god for consistency)


•  This is the table for all the databases Applica6on Uid lookup Table

Table Name Uid Column User Key Column

Blogs Not needed Not needed Not needed

Bookmarks PERSONLOGIN LOGINNAME PERSON_ID

Files USER_TO_LOGIN LOGIN_ID LOGIN_ID

Forum DF_MEMBERLOGIN LOGINNAME_LOWER MEMBERID

Homepage LOGINNAME LOGINNAME PERSON_ID

Ac?vi?es OA_MEMBERLOGIN LLOGINNAME MEMBERID

Profiles EMPLOYEE PROF_UID_LOWER PROF_KEY

Communi?es MEMBERLOGIN LOWER_LOGIN MEMBER_UUID

Wikis USER_TO_LOGIN LOGIN_ID USER_ID


•  This is the table for all the databases Applica6on Last Logon table

Table Name Uid Last Logon

Blogs ROLLERUSER USERNAME LASTLOGIN

Bookmarks PERSON PERSON_ID LASTLOGIN

Files USER ID LAST_VISIT

Forum MEMBERPROFILE MEMBERID LASTLOGIN

Homepage PERSON PERSON_ID LAST_UPDATE

Ac?vi?es OA_MEMBERPROFILE MEMBERID LASTLOGIN

Profiles PROFILE_LAST_LOGIN PROF_KEY LAST_LOGIN

Communi?es MEMBERPROFILE MEMBER_UUID LASTLOGIN

Wikis USER ID LAST_VISIT


•  Create a Domino Database with a form called “User” and following fields: –  Ac?vi?es_LASTLOGIN, Name, Blogs_LASTLOGIN,

Communi?es_LASTLOGIN, Dogear_LASTLOGIN, Files_LASTVISIT, Forum_LASTVISIT, Homepage_LASTUPDATE, Profiles_LASTLOGIN, Uid, Wikis_LASTVISIT

•  And a view to show these


•  Add a LotusNotes connector to the assembly line and connect it to your database using diiop.

•  Set the mode to “AddOnly”


•  Create the following output map •  The reason for not having the value as is in the le{ column is

because the value you get from db2 is in java.sql.date format, we need to make sure we get the string

Export users last logon date – How to •  To dump to a csv file add a File System Connector and select

csv as parser. Add the header fields to the Field Names and enable the write header

•  Set “;” as your seperator

Export users last logon date – How to Now we need to set the file loca?on and file name. We want to make this dynamic so we can schedule the script. File loca?on will be defined in the property file. Use the following javascript to define the filename and loca?on var srcPath=system.getTDIProperty("Cnx", "export_path")

var stDateStamp=system.formatDate((new Date()),"yyyyMMdd");

var outFile=srcPath + system.getTDIProperty("Cnx", "export_filename") + stDateStamp + ".csv"; return outFile

Export users last logon date – How to •  For the csv file we can output in the original format, no need

to transform to String as the parser will do this for us.

Community membership through a Domino application - Example

Community membership -‐ Example

Community membership – How to The workflow is as follows: 1.  Iterate through all Community entries in the Notes DB 2.  Create Community if it is a new Community

•  Check if it is a new community •  Create Community Atom entry •  Call/Reply request to the Communi?es API •  Get the Uuid of the new Community & write it back to the Notes DB

3.  Add missing members to every Community •  Iterate through all members found in the Community entry (from the

Notes DB) and look if user is not a member in the Community member feed

•  Create member Atom entry •  Send the member Atom entry to the Communi?es API

4.  Add missing Owners (same steps as for member adding)

Community membership – How to 1.  Iterate through all Community entries in the Notes DB:

Just use Lotus Notes Connector in iterator mode, again this is easy.

You don’t need a running HTTP task on Domino if you use the DIIOP IOR string as Server IP Address!

Community membership – How to 3.  Send the Wiki page Atom document to the Wikis API (HTTP

client connector): •  This is good documented (no joke)

hmp://www-‐10.lotus.com/ldd/appdevwiki.nsf/dx/Upda?ng_a_wiki_page_ic50

Community membership – How to 2.  Create Community if it is a new Community

•  Check if it is a new community


•  Create Community Atom entry var atom_community_entry = '<?xml version="1.0" encoding="UTF-‐8"?><entry xmlns="hmp://www.w3.org/2005/Atom" xmlns:app="hmp://www.w3.org/2007/app" xmlns:snx="hmp://www.ibm.com/xmlns/prod/sn"><?tle type="text">' + work.Community_Name + '</?tle><content type="html">' + work.Descrip?on + '</content><category term="community" scheme="hmp://www.ibm.com/xmlns/prod/sn/type"></category><snx:communityType>' + work.Access + '</snx:communityType></entry>';


•  Call/Reply request to the Communi?es API

This user needs the admin security role for the Communi?es app! (WAS Admin Console)


•  Get the Uuid of the new Community & write it back to the Notes DB

Community membership – How to 3.  Add missing members to every Community

•  Get the Community member feed (received with HTTP client connector)

This will create a request to following URL: …/communi?es/service/atom/community/members?communityUuid=$uuid&role=member


•  Iterate through all members found in the Community entry (from the Notes DB) and look if user is not a member in the Community member feed


•  Create member Atom entry through script:

var atom_member_entry = '<?xml version="1.0" encoding="UTF-‐8"?><entry xmlns="hmp://www.w3.org/2005/Atom" xmlns:app="hmp://www.w3.org/2007/app" xmlns:snx="hmp://www.ibm.com/xmlns/prod/sn"><contributor>¨<email>' + work.InternetAddress + '</email><snx:role>member</snx:role></contributor><snx:role component="hmp://www.ibm.com/xmlns/prod/sn/communi?es">member</snx:role></entry>’;

Community membership – How to 3.  Add missing members

to every Community •  Send the member

Atom entry to the Communi?es API (HTTP client connector)

URL on next page

This user needs the admin security role for the Communi?es app! (WAS Admin Console)


•  Send the member Atom entry to the Communi?es API (HTTP client connector)

This will create a request to following URL: …/communi?es/service/atom/community/members?communityUuid=$uuid

Community membership – How to 4.  Add missing Owners (same steps as for members)

var atom_owner_entry = '<?xml version="1.0" encoding="UTF-‐8"?><entry xmlns="hmp://www.w3.org/2005/Atom" xmlns:app="hmp://www.w3.org/2007/app" xmlns:snx="hmp://www.ibm.com/xmlns/prod/sn"><contributor>¨<email>' + work.InternetAddress_Owner + '</email><snx:role>owner</snx:role></contributor><snx:role component="hmp://www.ibm.com/xmlns/prod/sn/communi?es">owner</snx:role></entry>’;

Community membership – How to •  Final assembly line

Thank You!

http://www.webgate.biz

slideshare.com/kbild

linkedin.com/in/kbild

twitter.com/kbild

kbild.ch

slideshare.com/palmke

linkedin.com/in/wannesrams

twitter.com/wannesrams

wannes.rams.be

Addendum •  IBM Tivoli Directory Integrator Users Group

hmp://www.tdi-‐users.org/ •  IBM TDI Google Group

hmps://groups.google.com/forum/#!forum/ibm.so{ware.network.directory-‐integrator

•  Loca?on of diiop_ior.txt (created during the ini?al load of the DIIOP task) domina_data/domino/html

Presentations & Public Speaking

The lazy administrator, how to make your life easier by using tdi to automate your work - soccnx