perl usage at database applications

Embed Size (px)

Citation preview

  • 1. Perl DBI usage at report building Joe Jiang [email_address]

2. Perl Language Practical Extraction and ... Reporting Language 3. Perl Programmer Laziness Impatience Hubris (Boldness) 4. DBMS Database Management (descide) System 5. DBI.pm The gate to Management level programming 6. Back to reporting The man in breau The game they played The suggestion I once made on OLAP 7. Chemical Perl (Gas-ware) PHP for website Python for architecture Ruby for rails Perl for what? 8. Reports again DMBS is the container SQL is for query Perl is the formater 9. Give me Excel Copy HTML table into Excel Or, name HTML as .xls Use some IDE tool? 10. Toolset basic $ echo 'select SESSION_ID, HIT_ID from hit where USERNAME=? and ACCESS_TIME between to_date(?,?) and to_date(?,?)' > session.sql 11. It worked $ bofc < session.sql user-123 201106122345 YYYYMMDDHH24MI 201106122357 YYYYMMDDHH24MI | wc -l SESSION_IDHIT_ID 12 12. And ... $ bofc < session.sql user-123 201106122345 YYYYMMDDHH24MI 201106122357 YYYYMMDDHH24MI | sort SESSION_IDHIT_ID 201106122340461772475805 201106122340461772475807 201106122340461772475809 201106122340461772475811 201106122340461772475813 ... 13. Highlights

  • works with wc -l & sort (Laziness)

14. works with parameter binding (Boldness) 15. purify the SQL with limited punctuaction (Impatience) 16. Camel coding PERL5LIB=/proj/Web/EMIS/Modules perl -MEMIS::Database -le '($s= do{$d=EMIS::Database::gethandle(q(CDB)); $d->do(q(begin; nls_format_setting; end;)); $d}->prepare_cached(join q(),))->execute(@ARGV); $,=qq(t); print STDERR @{$s->{NAME}}; print @r while @r=$s->fetchrow_array; $d->disconnect' $* 17. Hints

  • PERL5LIB=/proj/Web/EMIS/Modules -MEMIS::Database

18. -le ' ... ' 19. $s = do { $d=...; ...; $d } -> prepare_cached( join q(),) 20. ( $s = ... ) -> excute ( @ARGV ); 21. $, = qq( t ); 22. print STDERR @{$s->{NAME}}; 23. print @r while @r=$s->fetchrow_array; 24. emp #!/usr/bin/perl -l useDBI; $s=do{ DBI-> connect ( q(DBI:mysql:database=employees) ,q() ,q() ); }->prepare_cached(join ( q() ,) ); $s->execute ( @ARGV ); $, = qq( t ) ; print STDERR @{$s->{ NAME }} ; print @r while @r = $s->fetchrow_array ; 25. Work Horse But I want query more than one DB ... Can you send me another column? Make a comma seperated list in one column ... 26. Report Driven Report

  • binding variables no longer in @ARGV

27. SQL no longer in STDIN 28. Here comes loop 29. Running faster by prepare

  • $s=$d->prepare_cached(do { open(SQL, q(execute(@F); 31. @r=$s->fetchrow_array; 32. @x = map {q()} @r unless @x; 33. IO::Handle::autoflush STDOUT 1; 34. emp-for #!/usr/bin/perl -wlan useDBI; useIO::Handle; BEGIN { $s = do { DBI-> connect ( q(DBI:mysql:database=employees) ,q() ,q() ); }->prepare_cached( do{open ( SQL ,q(finish ;$d->disconnect ;close SQL ; } $s->execute ( @F ); if( @r = $s->fetchrow_array ) { print @r ;@x=map { q() } @r unless @x }else{print @x} 35. How to list last deptartment? $ echo 'select emp_no from employees limit 3'|emp 2>/dev/null 10001 10002 10003 $ echo 'select emp_no from employees limit 3'|emp 2>/dev/null| emp-foremp_dept.sql d005 d007 d004 36. Highlights
    • First time in Perl script

    37. xargs for columns 38. Easier than outer join 39. Update? It's easy 40. Demo data replace bofc < x.sql | tee x.bak | do-bofc-for.pl y.sql

    • Backup and update in one line

    41. Avoid the "no where" update fault 42. Save a lot of coding with perl -lan 43. Or compose the code for debugging purpose 44. Extensions Merge Rotate Wrap 45. merge-tab

    • @f= map { IO::File->new($_) } @ARGV;

    46. map { $_->getline } @f 47. join (qq(t), map { m{(.*)} && $1 } ... 48. print ... until m{^t+$} 49. row2col

    • perl -MList::MoreUtils=each_arrayref

    50. push @lines, [@F]; 51. END{ ... } 52. $ea=each_arrayref(@lines); 53. print join qq(t), @x while @x=$ea->(); 54. Demo usage How to describe a wide table? With sample data attached? 55. Columns $ echo 'select DATA_TYPE, CHARACTER_MAXIMUM_LENGTH from information_schema.columns where table_name=?'| emp employees | tee emp-desc DATA_TYPECHARACTER_MAXIMUM_LENGTH int date varchar 14 varchar 16 enum1 date 56. Demo Row $ echo 'select * from employees limit 1' | emp 2>&1 | row2col | teeemp-data emp_no10001 birth_date1953-09-02 first_nameGeorgi last_nameFacello genderM hire_date1986-06-26 57. And now? Run merge-tab over the two files $ merge-tab emp.data emp.desc emp_no10001int birth_date1953-09-02date first_nameGeorgivarchar 14 last_nameFacello varchar 16 genderMenum1 hire_date1986-06-26date 58. 2excel & 2csv

    • perl -MSpreadsheet::WriteExcel

    59. $s->write_unicode($.-1, $_, encode q(utf16), decode q(utf8), $F[$_]) 60. perl -MText::CSV_XS 61. 2zip

    • BEGIN { $z=Archive::Zip->new()}

    62. $z->addString($_, $F[0])->desiredCompressionMethod(8); 63. END { $z->writeToFileHandle(STDOUT) } 64. Thanks Any Question? http://weibo.com/saved2serve