22/10/2012
1
How to analyze thousands of complex crystal structures in a minute
Mario ValleUSPEX 2012 CECAM workshop – 22/10/2012
Tutorial planSTM4 components
and structure
CrystalFpSTM4
components
CrystalFpanalysis
examples
AVS/Expressconcepts
STM4quick tour
CrystalFpbatch
Logistic information
Login on the tutorial machines with:
Username: Password:
Test data files are under:
~tutoadmin/CSCSlib/stm4-examples/data
Run STM4 with:
$ cd ~tutoadmin/CSCSlib $ ./go
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Ready? Go!STM4 components
and structure
CrystalFpSTM4
components
CrystalFpanalysis
examples
AVS/Expressconcepts
STM4quick tour
CrystalFpbatch
22/10/2012
2
STM4 is a frameworkfor the development of
unusual and enhanced techniquesfor chemistry visualization
Start STM4 (and 2″ tutorial) Launch AVS/Express (with STM4 included):
$ express (Linux)
C:\> cd CSCSlibC:\CSCSlib> go.bat (Windows)
Go to Libraries STM4 Then go to the rightmost column (Full Apps) Drag the MolDisplayApp block in the gray area below marked
Applications In the window that pops up select the file format (e.g. PDB)
and read a file.(few examples in: CSCSlib/stm4-examples/data)
Mouse rotate. Middle Mouse Button+Shift: Zoom. +Ctrl: Pan. Reset with the button.
With the Modules drop down menu (top left) select Display Structure and change the molecule appearance to CPK.
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
STM4 functional areas
Libraries
Build areaView resultsModules
GUI
Viewer control
Access to the STM4 library
22/10/2012
3
STM4 modules (look at them)
A B C D E F G H I J K
A. Readers and writers: chemical formats, screenshots and moviesB. Bonds related modules: compute, count and editC. Modules that compute various derived structuresD. Atoms selection by various criteriaE. Trajectory related modulesF. Crystallography support modules: symmetries, replications, etc.G. Visualization of volumetric data like Gaussian cubesH. Structure display and related graphical objectsI. Probe, interaction and measurementsJ. Ready to run applications (plus display of the STM4 version)K. Specialized modules (like USPEX output analysis applications)
The prebuilt MolDisplayApp
STM4 applications
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
An application is composed by modules connected together
Colors of connection ports and lines are related to data type (only compatible types can connect)
Each module starts when it receives all the needed data
The user interface docks in the right place automatically STM4 pre built application MolDisplayApp
Another application example
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
22/10/2012
4
STM4 data types
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Molecule Data Type Standard AVS Field
Geometries
Screen Image
DisplayParameters
STM4 module documentation
STM4 resources
STM4 main pages http://mariovalle.name/STM4/
The STM4 modules documentationhttp://mariovalle.name/STM4/doc/STM4/
STM4 downloadhttp://mariovalle.name/STM4/download.html
The original STM3 paper (please cite it if you use STM4)M. Valle, STM3: a chemistry visualization platform,Zeitschrift für Kristallographie, vol. 220, no. 5-6, pp. 585-588, 2005
A user level AVS/Express coursehttp://mariovalle.name/AVS/introduction-to-xp.html
A (marketing) brochure on AVS/Expresshttp://www.avs.com/software/soft_t/avsxps.html
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
New topicSTM4 components
and structure
CrystalFpSTM4
components
CrystalFpanalysis
examples
AVS/Expressconcepts
STM4quick tour
CrystalFpbatch
22/10/2012
5
STM4 architecture has… …AVS/Express behind the scenes
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
What is AVS/Express?
A multiplatform development environment…
…for 3D visualization applications
…object oriented
…based on a visual programming paradigm
AVS/Express components
22/10/2012
6
Data Viewer
Module GUI
Editors (access to rendering parameters)
Status panel
Quick access
Viewer
Default library
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Libraries
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Viz techniques
More than 500 visualization techniques available
Readers available for a broad range of file formats
AVS/Express prototyping support helps to find the most useful technique
Any technique can be adapted to suit specific user requirements
22/10/2012
7
Network Editor: the build areaApplications are built dragging modules from the libraries to the Network Editor
Network Editor
Try to find the AVS/Express parts
Go to: Libraries Examples
Instantiate: Visualization OrthosliceBy dragging it to the empty area
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Dataflow architecture
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Dataflow architecture
Wait
Wait Wait
Wait
Wait
22/10/2012
8
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Dataflow architecture
Wait
Wait
Wait
Wait
Enter filename
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Dataflow architecture
Execute
Wait
Wait
Wait
Valid data
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Dataflow architecture
Wait
Wait
Wait
Valid dataNot valid
data
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Dataflow architecture
Wait
Wait
Enter filename
22/10/2012
9
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Dataflow architecture
Wait
Valid dataValid data
Execute
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Dataflow architecture
Execute
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
The result of this application Speak to the AVS/Express core
Object Manager
22/10/2012
10
Not only visualization: UI Kit AVS/Express modules with STM4
One of the advantage of STM4 is the possibility to mix AVS/Express standard modules with STM4 ones
AVS/Express modules in STM4 Save your work
Use extension .v
22/10/2012
11
Various AVS/Express editions
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Developer Edition Everything you need Modules to develop applications Standard modules can be opened
Visualization Edition Some modules not available Cannot create runtimes Cost less No differences if you are
an End User
STM4 works with both editionsSTM4 for USPEX tutorial - Mario Valle - 22/10/2012
Some AVS/Express resources
AVS homepage: http://www.avs.com/
Official documentation: http://help.avs.com/Express/
AVS forum: http://forum.avs.com/
AVS/Express built‐in examples
Visualization techniques book (in the AVS/Express manuals)
International AVS Center (IAC): http://www.iavsc.org/
IAC training material: http://www.iavsc.org/training
Patches, doc and examples: ftp://ftp.avs.com/pub/express/
Other resources on: http://mariovalle.name/AVS/
A user level AVS/Express course:http://mariovalle.name/AVS/introduction‐to‐xp.html
New topicSTM4 components
and structure
CrystalFpSTM4
components
CrystalFpanalysis
examples
AVS/Expressconcepts
STM4quick tour
CrystalFpbatch
STM4 modules
A B C D E F G H I J K
A. Readers and writers: chemical formats, screenshots and moviesB. Bonds related modules: compute, count and editC. Modules that compute various derived structuresD. Atoms selection by various criteriaE. Trajectory related modulesF. Crystallography support modules: symmetries, replications, etc.G. Visualization of volumetric data like Gaussian cubesH. Structure display and related graphical objectsI. Probe, interaction and measurementsJ. Ready to run applications (plus display of the STM4 version)K. Specialized modules (like USPEX output analysis applications)
22/10/2012
12
Pre-built applications
Ready to run applications MolDisplayApp Read and display a molecule. CrystalDisplayApp Read and display structures
with symmetries and unit cell replication. VolumeDisplayApp Structure plus volumetric data PickDisplayApp Measure structure TracesDisplayApp Trace the motion of atoms PickUnitCellApp Redefine the molecule unit cell
by corner picking and crops the structure inside the new unit cell.
FermiSurfacesApp and FermiBandsApp Display data from the EIGENFILE
MultiComponentApp to support the convex-hull method
Plus the STM version string
MolDisplayApp
Move to CrystalDisplay Add tetrahedra
22/10/2012
13
Add title, logo and background Measure a structure
Readers/writers
Readers and writer for various, static and dynamic, chemical file formats plus movies and images production. Read Structure Read a structure from file (Read
Structure No Bonds idem without computing bonds). Read Scalar and Read Vector Read a set of scalar
or 3D vector values from a file. Make Movie Capture a sequence of frames, one for
each different viewer content, and assemble them in a movie (AVI or MPEG).
Output Image Save a snapshot of the content of the current viewer.
Write Structure Write a structure to a file Write POV Ray Write the graphical scene for ray-
tracing
STM4 – formats supported
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
22/10/2012
14
Two static structures Display Rendering of chemical structures and other graphical objects.
Draw Structure and Draw Sw Structure Render structures Info Sets Editor Edit render characteristics Display Unit Cell Displays the borders of the unit cell. Set Rendering Mode To change the rendering of only some atoms Axis Glyph Display three arrows to mark the origin and the XYZ axis
directions, or the unit cell abc vectors. The glyph could also be offset from the origin.
Background Fade Provide a colored backdrop for which you can set the four corners colors.
Logo Create a logo image to be added to a viewer. Enclosing Ellipsoid Compute an ellipsoid that encloses the given set
of atoms. Time Legend Display a progress bar to show the current timestep or
current time for an animated molecule trajectory. Color Legend Display a legend for the structure representation
colors. Discrete Legend Color swatches for discrete data (eg. bond count) Smooth Tube Create a tube that connect all the given points. Error Tube Create a tube with a radius proportional to the values
associated to the connected points.
Display structure Bonds
Bonds computation, editing and management. Compute Bonds Compute atomic bonds. Count Bonds Add as atom data the atom's number
of bonds. Count H Bonds Count the number of H Bonds
present in the input structure. The count is added to the structure global data.
Compute Selected Bonds Add bonds for each atom individually plus coordination tetrahedra
Set Manual Bonds Manually add bonds between atoms.
22/10/2012
15
Compute Selected Bonds Compute derived structures
Modules that compute derived structures. Solvent Excluded Surface Compute the molecule
Solvent Excluded Surface. Backbone Compute a protein backbone. General Backbone Display a tube through user
selected atoms. Vector Glyph Display with arrows the vector data
associated with a given structure. Merge Values Add a new scalar or vector data to
atoms, bonds or globally to the given structure. Atoms Values Trasp Render atoms with transparency
dependent on the associated scalar value Particle Density Compute the average particle mass
density using uniform binning. Interpolating Plane Build the least square plane
interpolating a set of picked atoms (n ≥ 3).
Interactivity (probe & measure)
Interact with the structures: measure and editing. Measure Structure Access information about and
measure a displayed structure. Measure Closer Atoms Measure intra-atomic
distances between a given atom and the surrounding ones in a given range.
Pick Unit Cell Redefines the unit cell for the displayed structure by picking on four atoms.
Measure a structure
22/10/2012
16
Measure closer atoms Select subsets
Selector modules to extract a subset of the atoms. Select Atoms Select specific atoms. Select Fragments Select specific structure
fragments (i.e. residues). Threshold Data Select atoms whose
associated scalar value meets various criteria.
Delete Atoms Interactively delete atoms.
Data threshold Trajectories
Modules related to dynamic data. Accumulate Traces Accumulate traces
(accumulated positions) for specific atoms in a sequence of time steps.
Accumulate Global Data Accumulate a global scalar data in a form suitable for charting.
Interpolate Sequence Linearly interpolate a given number of positions between each pair of timesteps.
Velocity Mean Compute the average particle velocity using uniform binning.
22/10/2012
17
Accumulate atoms traces Volumetric data support
Rendering of volumetric data like Gaussian cubes.
Isosurface Create an isosurface for a given scalar value. Orthoslice Create an orthoslice for a given scalar value. Volume Render Volumetric render of a scalar valued
volume. Cube on Surface Assign to a surface the values of the
volume in which the surface is immersed in the positions occupied by the surface itself.
Cut Surface Cut a surface (eg. a Solvent Excluded Surface) with a plane.
Bicolor Surface Color a surface on one side with a user specified color leaving the other one with its original coloring.
Interpolate Volume Interpolate a given scalar volume adding points between original grid points.
Access volumetric data Volume rendering
22/10/2012
18
Crystallography support
Unit cells, periodic lattices and symmetries support. Replicate Lattice Replicate the structure unit cell along the three
base vectors. Apply Symmetries Compute new atoms positions from the space
group of the input molecule. Shift Unit Cell Shift the unit cell to re-center periodic phenomena
that are split between the opposite sides of the cell. Enlarge Unit Cell Add atoms outside the unit cell by different
criteria. Move Structure Translate and rotate a given structure modifying all
the atoms coordinates. Crop Structure Inside a geometric shape (cube, sphere, tube,…) Crop From Plane Remove atoms that dist more than a given value
from a plane. Miller Plane Crop the atoms using the plane identified by given
Miller indices. Draw Polyhedra Draw polyhedra around selected atoms X Ray Structure Factor Compute and display simplified X-Ray
structure factors. Remove Border Atoms Remove atoms that bonds with atoms
outside the unit cell. Merge Unit Cell Apply the unit cell of one structure to another one.
Crop Structure
Find symmetries Fermi surfaces
22/10/2012
19
Find enthalpy transitions Convex-hull method support
New topicSTM4 components
and structure
CrystalFpSTM4
components
CrystalFpanalysis
examples
AVS/Expressconcepts
STM4quick tour
CrystalFpbatch
CrystalFp applications
CrystalFp no energy
CrystalFp with energy
CrystalFp with energyand energy landscape
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Secret fantastic newCrystalFp version
22/10/2012
20
Load the structure file Load the energy file
Initialize CrystalFp, load data Compute fingerprints & distances
22/10/2012
21
Adjust grouping (if needed) Analyze results
Build scatterplot Scatterplot diagnostics
22/10/2012
22
Build energy landscape Structure alignment
Your turn!STM4 components
and structure
CrystalFpSTM4
components
CrystalFpanalysis
examples
AVS/Expressconcepts
STM4quick tour
CrystalFpbatch
Test with your data
22/10/2012
23
New topicSTM4 components
and structure
CrystalFpSTM4
components
CrystalFpanalysis
examples
AVS/Expressconcepts
STM4quick tour
CrystalFpbatch
Scripts differences
On WindowsRelease\cfp.exe ^‐‐verbose=1 ^‐‐elements=“Si O” ^SiO2‐24atoms.poscar ^SiO2‐24atoms.enthalpies
Script.bat
On Linux./cfp \
‐‐verbose=1 \‐‐elements=“Si O” \SiO2‐24atoms.poscar \SiO2‐24atoms.enthalpies
Script.sh
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
Usage:CrystalFp [options] POSCARfile [ENERGIESfile]
‐v ‐‐verbose (optional argument)Verbose level (if no argument, defaults to 1)
‐? ‐h ‐‐help (no argument)This help
‐t ‐‐elements (required argument)List of chemical elements
‐es ‐‐max‐step ‐‐end‐step (required argument)Last step to load (default: all)
‐ss ‐‐start‐step (required argument)First step to load (default: first)
‐et ‐‐energy‐per‐structure (no argument)Energy from file is per structure, not per atom
‐e ‐‐energy‐threshold (required argument)Energy threshold
‐r ‐‐threshold‐from‐min (required argument)Threshold from minimum energy
‐c ‐‐cutoff‐distance (required argument)Fingerprint forced cutoff distance
‐n ‐‐nano‐clusters ‐‐nanoclusters (no argument)The structures are nanoclusters, not crystals
‐b ‐‐bin‐size (required argument)Bin size for the pseudo‐diffraction methods
‐p ‐‐peak‐size (required argument)Peak smearing size
...
Generate data./cfp \
‐‐verbose=3 \‐‐elements="Si O" \‐‐end‐step=50 \‐‐cutoff‐distance=30 \‐‐fingerprint‐method=0 \‐‐distance‐method=0 \‐‐grouping‐method=1 \‐‐grouping‐threshold=0.05 \‐‐remove‐dupl=map.dat \‐‐checkpoint‐dir=./chk \‐‐summary=summary.dat \‐‐fld‐fingerprints=fp.fld \‐‐fld‐distances=dist.fld \‐‐sorted‐distances=sorted.dat \‐‐analysis=25 \‐‐analysis‐file=analysis.csv \SiO2‐24atoms.poscar SiO2‐24atoms.enthalpies
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
22/10/2012
24
Visualize results using Rsource('readers.r')
# Read sorted distances and display on a log‐log chart
s <‐ readSortedDist('sorted.dat')
plot(s, type='l', log='xy', xlab='Distance', ylab='Cumulative frequency',main=bquote(paste(SiO[2], " 24 atoms")))
# Display distances
d <‐ readDists('dist.dat')
image(1:ncol(d), 1:nrow(d), d, col=rev(heat.colors(15)), xlab='Fingerprint1', ylab='Fingerprint2', main=bquote(paste(SiO[2], " 24 atoms")))
# Display all fingerprints and one specific fingerprint (n. 25) with reference lines
f <‐ readFp('fp.dat')
image(1:nrow(f), 1:ncol(f), f, col=rev(heat.colors(15)), xlab='Fingerprint', ylab='Distance',main=bquote(paste(SiO[2], " 24 atoms")))
plot(f[25,], type='s', xlab='Distance', ylab='Fingerprint value',main=bquote(paste(SiO[2], " 24 atoms")))
abline(h=0, lty=3)
abline(v=0:3*ncol(f)/3+1, lty=2, col='green')
# All corresponding quantities in a scatterplot table (removing Index and Step)
a <‐ readAnalysis('analysis.dat')
pairs(a[‐c(1,2)], pch='.', cex=4, col='blue')
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
It does also scatterplots!./cfp \
‐‐end‐step=50 \‐‐verbose=3 \‐‐elements="Si O" \‐‐fingerprint‐method=0 \‐‐distance‐method=0 \‐‐grouping‐method=1 \‐‐grouping‐threshold=0.05 \‐‐remove‐dupl=map.dat \‐‐scatterplot \‐‐scatterplot‐file=scatterplot.dat \‐‐diagnostic‐file=diagnostic.dat \‐‐scatterplot‐param iterations 200 \‐‐scatterplot‐param retry 4 \‐‐scatterplot‐param kind 0 \‐‐scatterplot‐param iterations 200 \‐‐scatterplot‐param retry 4 \‐‐scatterplot‐param kind 0 \‐‐scatterplot‐param diagnostic 1 \SiO2‐24atoms.poscar \SiO2‐24atoms.enthalpies
STM4 for USPEX tutorial - Mario Valle - 22/10/2012
source('readers.r')
# Prepare the color palette
n <‐ 256
pal <‐ heat.colors(n)
# Output the scatterplot
x <‐ readScatterplot('scatterplot.dat')
idx <‐ cut(x$value, seq(min(x$value), max(x$value),length.out=n+1), labels=FALSE, include.lowest=TRUE)
plot(x$x, x$y, xlim=c(‐1,1), ylim=c(‐1,1), col=pal[idx],pch=16, cex=1)
# Output the corresponding diagnostic chart
x <‐ readScatterplot('diagnostic.dat')
idx <‐ cut(x$value, seq(min(x$value), max(x$value),length.out=n+1), labels=FALSE, include.lowest=TRUE)
plot(x$x, x$y, xlim=c(0,1), ylim=c(0,1), col=pal[idx], pch=16, cex=1)
abline(0, 1, lty=1)
abline(0.1, 1, lty=3,lwd=2)
abline(‐0.1, 1, lty=3, lwd=2)
End of a nice day…
Thank you!BTW, I’m [email protected]
Geneva lake sunset