FPS gridification guide
From EGEE-see WIki
Contents |
Application description
One of the most important problems of seismology is definition of the fault mechanism parameters. Especially after the developments on broad-band seismometers MTI method became the most popular solution for the determination of the fault plain parameters. After 1990 a lots of analysis software were developed for MTI with many of them being often based on three main software packages: One of them was DMT_INV (deviator moment tensor inversion) developed by Ammon and Randall [1] that focus on regional scale solutions. The second one is TSBW_INV (teleseismic body-wave inversion) which is a software package developed by Kikuchi and Kanomori [2] for global scale solutions. The third one is TDMT_INV (time domain moment tensor inversion) package developed by Dreger [3] for regional solutions. All these software packages consist of two stages: the first step is calculation of synthetics seismograms and the second stage is application of MTI procedure.
For fault plain solution, the most recent application that is integrated with EGEE (Enabling Grid for E-science) is developed by Clévédé et al [4]. The main difference between Clévédé et al’s work and our FPS application is that while EGEE application is using teleseismic data, FPS is using regional data.
In this study, synthetic seismograms or Green functions are calculated using a frequency domain algorithm called FKRPROG that was developed for horizontally layered media by Saikia [5]. For MTI procedure FPS uses Dreger’s TDMT_INV [8] software. Parallelization is done by generating artificial seismogram level which is computationally intensive. FPS is parallelized by creating a process for each crustal model file at one of the seven different depths as shown in Figure 1.
Figure 1. FPS Workflow
Moment Tensor Inversion
Moment tensor, on a three-dimensional coordinate system, is the matrix form of a dipole force (seismic source) covering all nine possible force configurations. Here, the concept of dipoles force is two forces which are at the equal intensity and opposite direction. These dipoles forces in the form of matrices can be shown as follows [6]
Depending on M0 seismic moment and geographic coordinates, this equation can be written as:
Mxx = -M0 [sin(δ) cos(λ) sin(2Φ) + sin(2δ) sin(λ) sin²(Φ)] Mxy = M0 [sin(δ) cos(λ) cos(2Φ) + 0.5 sin(2δ) sin(λ) sin(2Φ)] Mxz = -M0[cos(δ) cos(λ) cos(Φ) + cos(2δ) sin(λ) sin(Φ)] Myy = M0 [sin(δ) cos(λ) sin(2Φ) - sin(2δ) sin(λ) cos²(Φ)] Myx = -M0[cos(δ) cos(λ) sin(Φ) - cos(2δ) sin(λ) cos(Φ)]
Here, δ is dip (the angle between the fault plane and the Earth's surface; Φ? is the strike (the angle between fault direction and geographic north); λ is rake (the angle between slip direction and the fault line).
Determination of the seismic moment tensor components (Mij) will be sufficient to calculate fault parameters (dip, strike and slip). In terms of moment tensor components (Mij), waveform seismic data (Sj), and synthetic seismograms (Aij) the problem can be defined by the following linear equation system:
In this equation i = 1, 2, 3, 4, 5 and j =1, 2, ..., 3n-1, 3n. (n is total number of all 3 component seismic data).
The solution of this equation system gives us the seismic moment tensor components that are directly related to faulting parameters.
Figure 2. Basic schema of MTI procedure
Development Details
Basic inputs of FPS application are 3 component seismic data, crustal model, earthquake coordinates, earthquake depth, coordinates of seismic stations, instrumental information and frequency range which is dependent on the magnitude of an earthquake. FPS application determines coordinates, depth and magnitude of an earthquake, and distance from the earthquake to stations, and azimuth value of stations by using the output of the HYPO71 [7] application. Figure 3 shows an example output of the HYPO71 program.
Figure 3. Example of HYPO71 output
DATE ORIGIN LAT N LONG E DEPTH MAG NO GAP … 070121 738 55.83 39-35.62 42-48.96 10.00 … CUKT 269.45 165 35 EP 2 98.59 98.10 0.49 … PTK 305.83 255 35 EPD3103.10102.77 0.33 … MALT 405.93 249 35 EPD3116.10115.60 0.50 … URFA 422.56 236 35 EP 3117.40117.73 -0.33 … …
FPS application generates crust models having 7 different depths by using the information provided in a file called the Model.ini file and by taking the earthquake centre determined by the HYPO1 program. Figure 4 shows an example Model.ini file.
Figure 4. Example of Model.ini file
[MODEL] nLayer=4 ;h(km),Vp(km/s),Vs(km/s),d(g/cm3),Qp,Qs Layer01=5.4,4.5,3.9,2.5,600,300 Layer02=26.2,5.9,4.3,2.6,1000,500 Layer03=56.6,7.8,5.7,3.0,1000,500 Layer04=400.0,8.3,6.1,3.4,1000,500 …
Kandilli Observatory and Research Institute used to compute Moment Tensor Inverse Solution by using crustal model files that have been formed by taking station distances in increments of 5 kilometres. Now, by using the output file of HYPO71, actual distances of stations are written to crustal model files. Since the regular 5 kilometre incrementation has been cancelled, run_filter and run_fksort programs that provide the initial data to TDMT_INVC program were also updated for using real distances of stations. An example crustal model file is shown in Figure 5.
Figure 5. Example of Crustal Model file
.F. 0 64 GREEN.2.1 4.0 4.00 1 512 1024 0.500 5 1 1 1 1 1 1 1 1 1 1 1 0 0.4000E+01 … 0.2500E+01 600.00 300.00 0.1400E+01 … 0.2500E+01 600.00 300.00 0.2620E+02 … 0.2600E+01 1000.00 500.00 0.5660E+02 … 0.3000E+01 1000.00 500.00 0.4000E+03 … 0.3400E+01 1000.00 500.00 2 0.4000000E+03 1.500000E+00 0 4 10000.0 30.0 1.9 1.7 269.45 0.0 8.0 305.83 0.0 8.0 405.93 0.0 8.0 422.56 0.0 8.0
A program called PREPDATA was developed converting waveform data in SAC format to a format used by the TDMT_INVC program. A JDL workflow was developed for parallelization. Every crustal model file which is automatically generated by FPS and the input files associated with crustal model files are sent to different worker nodes. Therefore, every computationally intensive Green function calculation is performed on different worker node. Each worker node calculates faulting components by using the Green function it generated. The faulting components that have been computed using the worker nodes are sent to a best solution finder worker node. Solution finder node finds the solution with the highest variance reduction and outputs it.
Automatic JDL Generation
In order to generate JDS workflows automatically a program was developed. Figure 6 shows an example workflow that was automatically generated. Here RunProcess.2.1.sh is responsible for running the processes in order.
Figure 6. Example of JDL workflow file
type = "dag";
VirtualOrganisation = "seismo.see-grid-sci.eu";
Rank = other.GlueCEStateFreeCPUs;
max_nodes_running = 8;
InputSandbox = "FPS.exe";
nodes = [
part0 = [
description = [
Type="Job";
JobType="Normal";
Executable = "/bin/sh";
Arguments = "RunProcess.2.1.sh";
StdOutput = "out.txt";
StdError = "err.txt";
InputSandbox = {
"2","MODEL.2.1","MODEL.2.1_mt_inv.in",
"RunProcess.2.1.sh","FPS.exe","b2s.par",
"Response_Coordinate.zip","Files.zip"
};
OutputSandbox = {
"MODEL.2.1.zip","out.txt","err.txt"
};
RetryCount = 3;
];
];
...
part7 = [
description = [
Type="Job";
JobType="Normal";
Executable = "FPS.exe";
Arguments = "4 7 MODEL.2.1 MODEL.2.2
MODEL.2.3 MODEL.2.4 MODEL.2.5
MODEL.2.6 MODEL.2.7 ";
StdOutput = "out.txt";
StdError = "err.txt";
InputSandbox = { root.InputSandbox,
root.nodes.part0.description.OutputSandbox[0],
root.nodes.part1.description.OutputSandbox[0],
root.nodes.part2.description.OutputSandbox[0],
root.nodes.part3.description.OutputSandbox[0],
root.nodes.part4.description.OutputSandbox[0],
root.nodes.part5.description.OutputSandbox[0],
root.nodes.part6.description.OutputSandbox[0]
};
OutputSandbox = { "BestSolution.zip",
"Others.zip","out.txt","err.txt"
};
RetryCount = 3;
];
];
];
dependencies = { { {part0, … , part6 },part7}};
An example JDL file which FPS program generates is shown in Figure 6. RunProcess.2.1.sh is a shell script that facilitates the execution of all the tasks in the JDL workflow. The file named 2 is the output of the HYPO71 application, MODEL.2.1 is a crustal model file, b2s.par is the input of the run_filter script, and MODEL.2.1_mt_inv.in is the input file of TDMT_INVC program. Response_Coordinate.zip file holds coordinate information about the stations and Pole Zero files about the station sensors.
File.zip file includes TDMT_INVC program, PREPDATA, FKRPROG, WVINT9, run_filter, run_fkrsort, tdmt.config.linux, and tdmt_redi_prepdata codes. Tdmt.config.linux does path definition for TDMT_INVC and PREPDATA. In addition, it does LFC catalog definition. FKRPROG is the program that generates green function. Run_fksort uses WVINT9 application, and it divides green function according to given depths. PREPDATA is responsible for filtering, sampling and relocation of data so that it is ready for the TDMT_INVC application.
In Figure 6, JDL file has 7+1=8 worker nodes. JDL file uses a DAG type workflow and seismo.see-grid-sci.eu virtual organization. Thanks to RunProcess.2.1.sh, every node knows which files it will process. Worker nodes from 0 to 6 compute faulting components in the above structure and send the results to part7. After all the part0 to part6 processes are finished, part7 process finds the solution which has the biggest variance reduction and transfers the solution to the output environment. Dependencies among the processes are specified in the "dependencies" section at the end of the workflow. Here, it is stated that process part7 is dependent on the processes part0... part6 and it hence will be started after all these have completed.
References
[1] RANDALL, G.E., 1994, Efficient Calculation Of Complete Differential Seismograms For Laterally Homogeneous Earth Models. Geophysical Journal International, 118, pp. 245–254.
[2] KIKUCHI, M., KANAMORI, H., 2003, Note on Teleseismic Body-Wave Inversion Program, http://www.eri.u-tokyo.ac.jp/ETAL/KIKUCH.
[3] DREGER, D.S., 2002, Time-Domain Moment Tensor Inverse Code (TDMT_INVC) Release 1.1, http://www.seismo.berkeley.edu/~dreger.
[4] CLÉVÉDÉ, E., • WEISSENBACH, D., • GOTAB, B., 2009, distributed jobs on EGEE Grid infrastructure for an Earth science application: moment tensor computation at the centroid of an earthquake. Earth Sci Inform 2:97–106.
[5] SAIKIA, C.K., 1994, Modified frequency-wave number algorithm for regional seismograms using Filon’s quadrature; modelling of Lg waves in eastern North America, Geophysical Journal International, 118, 142-158.
[6] JOST, M., HERRMAN, R.B., 1989, A Student's Guide to and Review of Moment Tensors. Seismological Research Letters. 60, 37-57.
[7] LEE, W. H. K., LAHR, J.C., 1972, HYPO71: A Computer Program for Determining Hypocenter, Magnitude, and First Motion Pattern of Local Earthquakes. Open File Report, U. S. Geological Survey, 100 pp.
[8] DREGER, D., ROMANOWIC, B.G. CLITHEROE, G., PEGGY HELLWEG, P., STEVENS, J., 2001, Development Of Automated Moment Tensor Software At The Prototype International Data Centre. 23rd Seismic Research Review: Worldwide Monitoring of Nuclear Explosions. October 2-5, 2001
