37
Kevin Cowtan, [email protected] “Pirate applications”... Pirate: phase improvement software Brigantine: bias removal software Unnamed application Kevin Cowtan [email protected]

Kevin Cowtan, [email protected] CCP4 March [email protected] Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Embed Size (px)

Citation preview

Page 1: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

“Pirate applications”...

Pirate: phase improvement software Brigantine:bias removal software Unnamed application

Kevin Cowtan

[email protected]

Page 2: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Pirate

Status: Statistical phase improvement software. Included in CCP4 version 6.0.

Updated in 6.0.2

Includes: Basic phase improvement:

Effective handling of order/disorder.

NCS averaging: Automated NCS determination from heavy atoms and density. Per-pair weighted NCS averaging.

Page 3: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Pirate

Status (2006): Phase improvement works well:

Slower than ‘dm’ Sensitive to the quality of the input HL coefficients

NCS: When it works, it works well. Buggy: works in < 10% of cases!

Why? Because at this meeting last year, I was told that

Buccaneer was more important.

Page 4: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Pirate

Status (2007): Automation further improved:

Log-likelihood weight now calculated automatically by

optimisation of the log-likelihood gain. Next version will also incorporate a better algorithm for

estimating the cell composition.

User interface improved: User no longer has to provide a reference structure

(unless they want to). GUI simplified.

Page 5: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Pirate

Status (2007): NCS code heavily updated. Now works fully

automatically in 30-50% of cases. Outstanding issue with NCS along special directions. When it doesn’t work, it does no harm. Minor tuning still required. (Timeconsuming!)

Page 6: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Pirate

Future:

Further NCS improvements.

Use of an input model of electron density map to

provided an additional source of information Integration with buccaneer.

Page 7: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Brigantine

Brigantine is a bias removal program:“Bias Removal In General Allows New Trace In New Density”

Mainly aimed at tackling MR bias problem, but uses

ideas and code from Pirate, but has applications to

validation and maybe density modification as well.

Page 8: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Brigantine

Status:

Currently performs 3 calculations: Simple composite omit map.

Fast: seconds to a few minutes.

Sigmaa-weighted composite omit map. Fairly fast: minute to tens of minutes

Pirate composite omit map. Fairly slow: tens of minutes to a few hours.

All of these are model-free, i.e. Can be used before

model building.

Page 9: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Brigantine

Pirate composite omit map. Related to a similar idea in Resolve

(CCP4 study weekend 2007)

But does not involve any density modification. i.e. It can be used in combination with arbitrary density

modification methods. Maybe even recycled with several

different density modification methods.

Code works. One tunable parameter needs to be estimated. It is not clear if the results are useful.

Page 10: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Brigantine

Applications: After MR

(to reduce model biad)

Between different density modification procedures. (speculative)

As part of iterated density modification/model

building procedure. (as validation)

In model completion (validation, ligands, etc.)

Page 11: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Unnamed application

An unnamed program to provide an improved method

for perturbing a model to reduce bias after

refinement.

An alternative to simulated annealing.

Uses: Unbiasing a free set. Precursor to Brigantine.

Page 12: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Tools

Fast anisotropic scaling

(needs tuning)

Omit maps

Sigma-a maps

Refinement?

Page 13: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Buccaneer, Coot, and low resolution

Developments in buccaneer and coot

for model building, and application

to low resolution problems.

Kevin Cowtan

[email protected]

Page 14: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Buccaneer: Method

Statistical model building software based on the use

of a reference structure to construct likelihood

targets for protein features.

Page 15: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Buccaneer

Status (2006): Successful tool for protein chain tracing, even at

low resolutions.

Tasks (2006): Implement sequence assignment/docking. Implement recycling for model completion Iterate with refmac

Page 16: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Buccaneer

Status (2007): Sequence assignment/docking implemented. Recycling for model completion implemented. Iteration with refmac implemented.

In an ad-hoc script, without GUI

Page 17: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Buccaneer

Sequence assignment/docking:

Use the same method for classifying side chains by type as

for finding C-alphas in the first place, but with a

difference target function:

ALA CYS HIS MET THR ... x20

Finding:

Classifying:

Page 18: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Buccaneer

Recycling for model completion implemented. “Lateral growing”: grow sideways from existing

chain fragments by looking for new C-alphas at an

appropriate distance “sideways” from the existing

chain. Fast recycling within Buccaneer, or slower recycling

with other programs, e.g. refmac.

Page 19: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Buccaneer

Steps: (bold are new since 2006) Find C-alphas Grow into chains Join the chains Pre-sequence loop building (up to 2 C-alphas) Assign sequence Correct insertions/deletions Filter based on poor density Remove clashing chains Build side chains

Page 20: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Buccaneer

The positive: Successfully applied to 2 real unknown

structures with good phases at ~3.5A. More

than half the model built and sequenced. Successfully applied to 4.0A truncated data. First citation this month!

The negative: Model completeness still not as good as

Resolve. What about bad phases at higher resolutions?

Page 21: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Page 22: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Page 23: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Page 24: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Coot

Coot developments in model building:

Improved helix placement Improved handling of low resolution maps Handle ‘UNK’ for use with Buccaneer Developments towards strand placement Future integration of Buccaneer functionality

Page 25: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Coot

Improved helix placement: Helix built both ways and refined without Cb. Correct orientation determined from Cb density.

Low resolution: Add terminal residue does RSR Auto-fit sidechain does RSR Secondary structure restraints

Page 26: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Coot

Secondary structure: Helix Strand

Page 27: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Coot

Strand placement New algorithms are being examined for both helix

and strand placement. First attempt:

Use skeleton and find where ridges cross a sphere

containing the feature. Faster than current method for helices.

Page 28: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Coot

Integration of Buccaneer features: 'UNK' residue type handling. Buccaneer sequencing has been exported to a

standalone ‘cootaneer’ library. For speed, this has to work without a reference structure. Use pre-calculated resolution dependent libraries. Just needs integrating.

Buccaneer growing may be added in future. An alternative engine for current ‘Add terminal residue’? Or an auto-grow algorithm.

Page 29: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Wiki and CCP4

Wiki as a tool to organise and

expand CCP4 documentation

and mailing list answers.

Kevin Cowtan

[email protected]

Page 30: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

What is a Wiki? A website which anyone (or anyone authorised)

can edit. Wikipedia is an example:

Page 31: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Why Wiki? Much easier to update than a website...

Edit through the web browser, even remotely. Automatic markup (most of the time). Semi-automatic linking and cross referencing.

Collaborative... Many people doing the bits they know best when it is

convenient to them.

Page 32: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Wiki Features Build in version control.

Track and revert changes.

Access control for admin, writing, (reading). Initially wikis were open, these days must register. Per-page access control as well as site wide.

Many allow pages with discussion areas. A page can be read only but allow comments.

Page 33: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

How could CCP4 use one? Take good bits out of the program documentation,

manual, bulletin board discussions, and gather and

organise them. Turn bulletin board discussions into FAQ pages. Host frequently changing info:

Contact details Useful web resources Problem pages

Page 34: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Case studies:

~2002: Tried to use one to document eHTPX

collaboration. Failure: Unfamiliar. People forced to contribute.

~2003: Implemented at York for local documentation

pages. Success: Widely read. A smaller group contribute. Enough enthusiastic contributors to make it work. There must be a ‘critical mass’ of contributors.

Volunteers: EJD, JPT, KDC

Page 35: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

YSBL Wiki: 135 pages from 7 people

Page 36: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Deploying a Wiki

Issues: The server is web-facing Wikis are regularly attacked

Software updates must be applied very frequently Better to have a dedicated machine outside the firewall

Choice of wiki software: MoinMoin, Twiki, Mediawiki, etc... Language, database or flat file, access control, version

control, comments, images, math, plugins, search, email

notification of changes, content export, etc...

Page 37: Kevin Cowtan, cowtan@ysbl.york.ac.uk CCP4 March 2007cowtan@ysbl.york.ac.uk Pirate applications... Pirate:phase improvement software Brigantine:bias removal

Kevin Cowtan, [email protected]

CCP4 March 2007

Deploying a Wiki

Issues: Alternative: Wiki hosting sites:

wikia, wikidot, wikispaces. Some are free, or $5-$50/month. Some display adverts Some demand content be licensed, e.g. CC, FDL

Issues: Convenience: can I be auto-logged in when I arrive? Can the content be exported if we need to move?