WRITING COMMAND-LINE TOOLS WITH IRONPYTHON AND VISUAL STUDIO Including PowerShell, and F# Noah Gift

Talk Objectives

¤ Freak you out a bit. ¤ Teach you something you

didn’t know. (Not covered in any book).

¤ Be controversial. ¤ Entertain you.

Monty Hall Problem

¤ You are invited to a game show.

¤ Game show host shows you three doors.

¤ Behind two doors there is a goat, and behind one door is a $50,000 dollar sports car.

Monty Hall Problem ¤ You choose a door. ¤  Instead of opening your

door. The game show host then opens up another door, to reveal a goat.

¤ Do you switch your choice?

Monty Hall Problem

¤ You always switch doors: 2/3 probability versus ½ probability.

¤ Bayes Rule shows us this: (Think Spam classification).

Monty Hall Takeaway

¤ This isn’t intuitive to most people

¤ The human brain’s intuition is often broken.

¤ What else is your brain wrong about?

Windows Is The Great Satan?

¤ Command-line is feeble ¤ People on twitter say it sucks ¤ Nothing COOL ever happens

on Windows ¤ This isn’t “their” conference. ¤ What if your brain’s intuition is

wrong about this too?

I Used To Hate Windows ¨  I Grew Up ¨  New Philosophy: Write

Code in any language, in any environment.

¨  If it is innovative, I don’t care who made it.

¨  Sorry Stallman, we disagree.

What do most developers and admins think of when they hear Linux?

¨  Power ¨  Flexibility ¨  Bash ¨  SSH

¨  The Shell ¨  Command-tools ¨  Man Pages ¨  Awk/Sed

What Is The Unifying Theme Around This?

¨  String Output is King ¨  Small Tools ¨  Flexibility ¨  Geek Power.

¨  Tim O’Reilly in Unix Power Tools, “A new user by stringing together simple pipelines….”

The Hardest Part of Writing a *Nix Command-line tool in Python

¨  Remote Administration: Pyro vs SSH vs ???

¨  Dealing with *Nix tools that pass you back unstructured text.

¨  Religious war in your company over using optparse, getargs, or argparse

Python/Unix Command-line Tools: The End Game

¨  Remote Object Invocation (Pyro, SSH, Fabric)

¨  Security (SSH, SSL) ¨  Automation ¨  Application API ¨  Event Management

(Pyro) ¨  Transactions (Mercurial?)

How To Write A Classic IronPython CLI

¨  I needed to upgrade 100’s of thousands of lines of C# to .NET 4.0.

¨  I wrote a quick and dirty Command-line tool in IronPython.

Project Upgrader: Validate Version

Step 1  def  validate_vsver(slnfile):  

       """Validates  whether  a  .sln  file  has  already  been  upgraded  to  VS  2010"""          try:  

               project_file  =  open(slnfile)                  for  project_version_line  in  project_file.readlines():  

                       if  "Visual  Studio  2010"  in  project_version_line:  

                               log.debug("Found  VS  2010  project  line:  %s"  %  project_version_line)                                  log.debug("Project  file  already  upgraded....skipping:  %s"  %  slnfile)  

                               return  False  

               return  True          finally:  

               project_file.close()  

Project Upgrader: Walk A Tree

Step 2 def  walk_tree(path,  extension="sln",  version_control_type=".svn",  exclude_path=None):          """Walks  a  tree,  if  there  is  a  match  to  an  extension,  it  returns  a  fullpath.                    Skips  version_control_type  directories,  which  by  default  is  set  to  .svn            """          for  dirpath,dirname,filename  in  os.walk(path):                  #The  walk  should  skip  version  control  directories                  for  dir  in  dirname:  

Project Upgrader: Convert Project

Step 3  def  convert_project(filename):          """Converts  a  project  to  VS2010  using  deven  via  .NET  Process  Class"""          p  =  Process()          p.StartInfo.UseShellExecute  =  False          p.StartInfo.RedirectStandardOutput  =  True          p.StartInfo.RedirectStandardError  =  True          p.StartInfo.FileName  =  """C:\\Program  Files  (x86)\\Microsoft  Visual  Studio  10.0\\Common7\\IDE\\devenv.exe"""  print  "Stdout:  %s|  Stderr:  %s"  %  (stdout,  stderr)          p.WaitForExit()          return  p.ExitCode  

Project Upgrader: Convert Tree

Step 4  def  convert_tree(path,  exclude_path):          """Recursively  modifies  a  tree  of  sln  files,  if  they  haven't  been  upgraded  to  VS  2010"""                    for  file  in  walk_tree(path,  exclude_path=exclude_path):                  if  not  validate_vsver(file):                          #we  should  pass  upgrading  if  validate_vsver  returns  False                          continue                  exit_code  =  convert_project(file)                  log.info("%s,  %s"  %  (exit_code,  file))  

Project Upgrader: Run method

Step 4    def  run(self):                  args,  parser  =  self.options()                  if  args.path:                          exclude  =  None                          if  args.exclude:                                  exclude  =  args.exclude                                  log.info("Exclude  set:  %s"  %  exclude)                          log.info("%s  Running  VS2010  upgrade  on  tree:  %s  |  excluding  dirs  named:  %s"  %  (time.ctime(),  args.path,  exclude))                          convert_tree(args.path,  exclude_path=exclude)                                  else:                          parser.print_help()

Python is like Kudzu

¤  Its strength is that it can embed itself into any project, any operating system.

¤ Not always the best tool, but often good enough.

Who Is The Target Audience For Command-line Tools?

¤ Systems Administrators ¤ Power Users ¤ Developers ¤ Systems Integrators/OEM ¤ Young Geeky Stallman

What Does It Really Mean To Write A Command-line tool on Linux

¤ You have an executable that runs in the context of, most likely, Bash.

¤ You output a string, which you can then pipeline into our Unix tools, sed, awk, grep.

¤  If your nice you create –help

What Does It Really Mean To Write A Command-line tool on Modern Windows

¤  It is going to run inside of PowerShell. PowerShell is your bash.

¤ You have a unified interface to .NET and every possible high level automation API you can think of.

¤ PowerShell tools expect OBJECTS not strings.

PowerShell is Secure By Default ¤ Default File

Association is Notepad or ISE.

¤ Remoting is disabled.

¤ No execution of scripts.

¤ Script Signing

Learning To Properly Write IronPython Command-Line Tools: “A two minute affair”

¤ Actually no, not even close. ¤ Forget “most” of what you

know from linux. ¤ PowerShell changes

everything.

PowerShell Object Passing Changes How Shells Work

¤ Fundamental change to understand

¤ You need to understand this to write command-line tools on Windows

¤ Game changer

We’re not in a flyover state anymore

¤ String output is lame! ¤ PowerShell wants objects ¤ What to do…what to do…

Fight the ecosystem and you will die

¤ Command-line tools should run in PowerShell not cmd.exe.

¤ The bash/linux CPython way, won’t cut it here.

¤ You must think the way .NET thinks.

¤ Everything takes and returns an Object.

PowerShell Endgame: We did EVERYTHING for you.

¤ Encrypted, paranoid secure, transactional, asynchronous remoting of serialized objects

¤ High level automation API to everything.

¤  Ignore at your own peril.

Writing a PowerShell Module In IronPython: The “real” way to write Command-line tools?

¨  Create a New Visual Studio Project ¨  Thinks Objects ¨  Use PowerShell SDK ¨  Write a Binary Module

Installing the PowerShell SDK

¤ Download the SDK ¤ Add a reference to the DLL in your Visual Studio Project

¤ C:\Program Files (x86)\Reference Assemblies\Microsoft\WindowsPowerShell\v1.0\System.Management.Automation

Finding and Loading PowerShell Modules

¨  Get-Module –all ¨  When you create a .dll in IronPython, could then

simply integrate it into the PowerShell workflow.

What is a Binary Module? ¤ A .NET assembly compiled against PowerShell

libraries. ¨  For IronPython folks, this is one way to integrate into

PowerShell ¨  In a lot of documentation it will refer to C# code. ¨  You can also write a PowerShell, non-compiled,

module.

Creating a Binary Module For Powershell in IronPython

¤ http://stackoverflow.com/questions/2094694/launch-powershell-under-net-4

¤ You need to tweak some registry settings to support .NET 4.0 in PowerShell.

Creating a Binary Module For Powershell in IronPython: Continued

¤ Should follow C# example code for best implementation advice.

¤ Note, Argparse, Optparse, etc, aren’t used. They don’t make sense.

¤ These are objects being passed into other cmdlets, and methods are being called on those objects.

IronPython PowerShell Module Hello World

Visual Studio Project:

Step 1

import  clr  clr.AddReference('System.Management.Automation')  print  'Hello  World'  

IronPython PowerShell Module Hello World

Visual Studio Project:

Step 2

¨  Build the project to create a .dll ¨  In PowerShell Import Module ¨  Import-Module MyModule.dll ¨  You can know interact with it from inside

of .Net. Is this a commandline tool???

Setting Up A PowerShell Profile For IronPython

¨  Just like in Bash or ZSH, aliases are your friend.

¨  Edit your $profile with the ISE, and add interactive IronPython to it.

Creating Standalone IronPython executables

¨  Make sure you can pass objects in and out of them.

¨  Use the IronPython on codeplex: Pyc-Python Command-line compiler

F#: The Elephant in the Room

¤ Type inference ¤  Immutable data structures ¤ Run as script or compiled ¤ Tail recursion ¤ Fully integrated with Visual

Studio ¤ Sexier then Python?

F# and Python: A Sexy Beast

¤ Mixing .dll files written in F# into IronPython modules, loaded into Powershell.

¤ Calling IronPython/CPython from inside of F#

¤ “A statically typed Python…” (Note, this will come up..just preparing you…)

Credit Some Excellent Books That Reference PowerShell

¤ PowerShell in Action, Manning (The Bible)

¤ PowerShell Cookbook, O’Reilly ¤  IronPython in Action, Manning

Conclusion

¤  PowerShell isn’t a competing language, it IS THE ECOSYSTEM.

¤ Windows Systems Admins are using PowerShell for everything

¤ Get your head wrapped around object based pipelines.

¤ Why doesn’t Linux have this?

Questions?

Code: https://bitbucket.org/noahgift/pycon2011-ironpython-cli