SEE-GRID File Management Java API
From EGEE-see WIki
This Wiki page is a part of SEE-GRID Gridification Guide. It is contributed by Belgrade University Computer Centre.
Contents |
Overview
SEE-GRID File Management Java API supports most of data and logical file management operations available in gLite environment through LFC and lcg-utils C APIs. It is developed and maintained at Belgrade University Computer Centre. This API was previously known as "LFC Java API" but the name was changed after its expansion with additional features.
The API is compatible with LCG 2.7.x and gLite grid middleware.
The initial version was created within SEE-GRID project during the gridification of Volumetric Image Visualization Environment (VIVE) application. Its further development within SEE-GRID-2 will hopefully help gridification of other applications.
One of artifacts of VIVE development is Web-based LFC Data Management Front End which uses our SEE-GRID File Management Java API. The paper "Grid Data Management Web Portal" describes the implementation of this portal and its features.
Another usage, providing higher-level features suitable for usage from command line or Java code, is Managing Sets of Files and Replicas Within LFC Catalog.
This API is also used in this GILDA tutorial, which includes several exercises that cover various aspects of API usage.
Available features
- Uploading files from UI to grid
- Downloading files from grid to UI
- Reading file information
- permissions
- GUID
- comment
- Listing file replicas
- File management operations
- replicating files (copying physical files from one SE to another)
- deleting files and file replicas
- unregistering files and file replicas
- Reading directory information
- permissions
- comment
- Listing directory contents
- Reading user and group ids for files and directories
- Reading user and group names for files and directories
- Reading filemode information as described in LFC C/C++ API manual
- Listing file/directory aliases
- Alias management operations (create/rename/delete)
- Reading file/directory date information
- Additional file management operations (rename/move)
- Directory management (create/rename/move)
- File/directory comment modification
- Setting of operational parameters using environment variables and configuration file
- Listing of SE from BDII with the ability to ignore some SEs set in a properties file
- Directory management (delete)
Planned features
- Basic FTS support: initiating FTS transfers and reading their status
- FPS using LFC and FTS
- The InputStream implementation for files will be developed if not provided by future versions of gLite middleware. Developers can also use GFAL API for Java (javadoc).
Installation and Configuration
The following APIs should be already installed and configured by your system administrator. This is done by following the gLite installation guides 1, 2. These APIs come with gLite middleware:
- GFAL C/C++ API
- lcg-utils C/C++ API
- LFC C/C++ API
Download the SEE-GRID File Management API. Unpack the archive into a directory present within CLASSPATH environment variable, or add the target directory to CLASSPATH.
When using this API, the used virtual organisation, BDII, and configuration file can be set through the parameters of LFCDataStorage constructor. If default constructor is used or null values are set as parameters, the actual used values are the following:
- If configuration file pathname is null, "repmngr.properties" is used, pointing to a file of the same name in current folder.
- The virtual organization name value is obtained from vo parameter. If null, the used virtual organization is obtained from the properties file in the current folder. If properties file or VO value in the file is missing, the value from environment variable LCG_GFAL_VO is used.
- The parameter bdii defines the host name and port number of BDII being used to obtain the list of available storage elements. It can be also set in properties file. If not, the value LCG_GFAL_INFOSYS of environment variable is used as default.
Please note that the usage of GFAL related operations, i.e. file content transfers between a storage element and local filesystem require LCG_GFAL_INFOSYS to bi consistent with the value in properties, if any.
If some of required parameters is not set, an exception will be thrown.
Properties file configuration:
- Set the value of property VO to your Virtual Organization name (for example: VO = seegrid)
- Set the value of properties bdii.host.name and bdii.host.port to point to the BDII
- Optionally, set the value of property SE.ignore.list - the list of SEs to be ignored when obtaining available SEs from the BDII. Use space to separate list items. (example: SE.ignore.list = se1.ignoredhost.org se2.ignoredhost.org)
Documentation
SEE-GRID File Management Java API documentation in javadoc format
Downloads
SEE-GRID File Management Java API
Version 1.3 (released 11/07/2007)
Source code, classes, properties file and examples (zip file, 87 KB)
Version 1.2 (released 10/30/2006)
Source code, classes, properties file and examples (zip file, 99 KB)
Version 1.1
Source code, classes, properties file and examples (zip file, 42 KB)
Version 1.0
Source code, classes, properties file and examples (zip file, 40 KB)
SEE-GRID Data Management Web Portal
Version 1.1 (released 06/13/2007)
Source code, classes, properties files and examples (zip file, 347 KB)
Examples
Listing directory content
import yu.ac.bg.rcub.grid.dataManagement.*;
public class LfcLs{
public static void main(String [] args){
if(args.length==0){
System.out.println("Usage: java LfcLs grid_directory\n");
System.exit(-1);
}
DataStorageInterface dsi = new LFCDataStorage();
DirectoryItem di = new LFCDirectoryItem(args[0],null,dsi);
ItemIterator iter = di.itemIterator();
while(iter.moreChildren()){
Item nextItem = iter.next();
System.out.print((new LFCFileMode(nextItem.getFileMode())));
System.out.print("\t"+nextItem.getUID());
System.out.print("\t"+nextItem.getGID());
System.out.print("\t"+nextItem.getSize());
System.out.print("\t"+nextItem.getName());
System.out.println();
}
}
}
Uploading file from UI to SE with LFC registration
import yu.ac.bg.rcub.grid.dataManagement.*;
public class copyAndRegister{
public static void main(String args []){
int numArgs = args.length;
if(numArgs!=4){
System.out.print("\nusage: copyAndRegister sourceFilePath ");
System.out.print("gridDestinationDir gridDestFileName SEName");
System.out.println();
System.exit(-1);
}
DataStorageInterface dsi = new LFCDataStorage();
DirectoryItem di = new LFCDirectoryItem("",args[1],null,dsi);
boolean success;
success = di.copyAndRegister(args[0],args[2],args[3]);
if(success){
System.out.print("File "+args[0]+" copied and registered as: ");
System.out.print(args[1]+"/"+args[2]+".");
System.out.println();
} else{
System.err.println("Unable to copy and register file.");
System.exit(-1);
}
}
}
Downloading files from SE to UI
import yu.ac.bg.rcub.grid.dataManagement.*;
public class download {
public static void main(String [] args) {
if(args.length != 3) {
System.out.println("Downloads file from grid.");
System.out.println(
"Usage: java download <lfc_file_to_download> <sourceSURL> <destinationFileName>");
System.exit(0);
}
DataStorageInterface dsi = new LFCDataStorage();
FileItem fi = null;
FileItem file = new LFCFileItem(args[0],null,dsi);
boolean success = file.download(args[1], args[2]);
System.out.println("Download successful: "+success);
}
}
Copying physical files from one SE to another (replicating)
import yu.ac.bg.rcub.grid.dataManagement.*;
public class replicate {
public static void main(String [] args) {
if(args.length!=2){
System.out.println("Replicate file to given SE.");
System.out.println("Usage: java replicate <lfnFileName> <target_SE>");
System.exit(-1);
}
DataStorageInterface dsi = new LFCDataStorage();
FileItem fi = null;
FileItem file = new LFCFileItem(args[0],null,dsi);
boolean success = file.replicate(args[1]);
System.out.println("Replication successful: "+success);
}
}
Contact
Dragan Okiljevic (oki AT rcub.bg.ac.yu)
