Data Access with GFAL

From EGEE-see WIki

Dear Ahmetk and Erayo

I am using this Wiki for communication since I do not have your e-mail addresses. Please take a look at SG_Using_file_replicas_and_RFIO:_UI_configuration,_rfiod,_usage_in_apps,_limitations_and_workarounds

You may wish to copy and update some contents from there or simply provide a link. On the original page RFIO is being discussed, but some details about dcap may be missing.

Branko

Introduction to GFAL

GFAL is a C API for performing input/output operations on grid files. GFAL functions have similar names with their corresponding i/o functions used on local files (having the prefix "gfal_"). GFAL accepts the following file naming conventions:

• Logical File Name (LFN)
• File Replica (SURL)
• Transport File Name (TURL)
• Grid Unique Identifier (GUID)

Before Starting (Preleminaries)

The following environment variables should be set correctly:

LCG_GFAL_INFOSYS should be set to point to a top BDII in the format <hostname>:<port> BDII read port is 2170.

Example:

> export LCG_GFAL_INFOSYS=bdii.ulakbim.gov.tr:2170
> echo $LCG_GFAL_INFOSYS
bdii.ulakbim.gov.tr:2170

LFC_HOST can be specified as the endpoint for the catalog (taking precedence over that published in the IS). If no endpoints are specified,the ones published in the Information System are taken.

Example:

> export LFC_HOST=grid02.rcub.bg.ac.yu
> echo $LFC_HOST
grid02.rcub.bg.ac.yu

LCG_CATALOG_TYPE should be set to lfc.

Example:

> export LCG_CATALOG_TYPE=lfc
> echo $LCG_CATALOG_TYPE
lfc

LCG_GFAL_VO should be set to the user VO.

Example:

> export LCG_GFAL_VO=seegrid
> echo $LCG_GFAL_VO
seegrid

GFAL C API

Functions

• Functions Used for File Management
  • gfal_create -create a new file or truncate an existing one-

int gfal_access (const char *path, int amode)

• • gfal_unlink -remove a file entry-

int gfal_unlink (const char *filename)

• • gfal_open -open a file-

int gfal_open (const char *filename, int flags, mode_t mode)

• • gfal_close -close a file-

gfal_close (int fd)

• • gfal_lseek -position a file-

off_t gfal_lseek (int fd, off_t offset, int whence)

• • gfal_read -read from a file-

int gfal_read (int fd, void *buf, size_t size)

• • gfal_write -write to a file-

int gfal_write (int fd, void *buf, size_t size)

• Functions Used for Directory Management
  • gfal_mkdir -create a new directory-

int gfal_mkdir (const char *dirname, mode_t mode)

• • gfal_rmdir -remove a directory-

int gfal_rmdir (const char *dirname)

• • gfal_opendir -open a directory-

DIR *gfal_opendir (const char *dirname)

• • gfal_closedir -close a directory-

int gfal_closedir (DIR *dirp)

• • gfal_readdir -read a directory-

struct dirent *gfal_readdir (DIR *dirp)

• Functions Used for Both File and Directory Management
  • gfal_access -check existence/accessibility of a file or a directory-

int gfal_access (const char *path, int amode)

• • gfal_chmod -change access mode of a file or a directory-

int gfal_chmod (const char *path, mode_t mode)

• • gfal_rename -rename a file or a directory-

int gfal_rename (const char *old_name, const char *new_name)

• • gfal_stat -get information about a file or directory-

int gfal_stat (const char *filename, struct stat *statbuf)

Example

The following C++ program first creates a file, writes on it and closes. Then file is reopened, read and closed. The name of the file should be given as an argument to the program.

#include <iostream>
#include <fcntl.h>
#include <stdio.h>
#include <fstream>

extern "C"
{
#include "/opt/lcg/include/gfal_api.h"
}
using namespace std ;

extern "C"{
int gfal_open (const char*, int, mode_t);
int gfal_write (int, const void*, size_t);
int gfal_close (int);
int gfal_read (int, void*, size_t);
}

void call_gfal_write(char *filename, size_t block_size);
void call_gfal_read(char *filename, size_t block_size);

int main (int argc, char **argv)
{
        char* filename;
        filename = argv[1];
        size_t block_size = 16;
        call_gfal_write(filename, block_size);
     	call_gfal_read(filename, block_size);
      	
	return 0;
}

void call_gfal_read(char *filename, size_t block_size)
{
	int FD; //file descriptor 
	int rc; //error code
        int array_size= block_size/sizeof(int);
	int* readValues= new int[ array_size ];

	if((FD = gfal_open ( filename, O_RDONLY,0 )) < 0)
	{ 
		perror ("error in gfal_open");
		exit(1);
	}
   	cout << "File is successfully opened\n";

	if ((rc=gfal_read (FD, readValues, block_size)) != block_size )
        {
		if (rc < 0) perror("error in gfal_read ");
		else cerr << "gfal_read returns "<< rc <<endl;
	}
   	cout << "File is successfully read\n";

	if ((rc= gfal_close (FD)) < 0)
	{
		perror ("error in gfal_close");
		exit(1);
	}
        
	cout << "Close successful ..."<<endl;
}

void call_gfal_write(char *filename, size_t block_size)
{
        int FD; //file descriptor 
	int rc; //error code
        int array_size= block_size/sizeof(int);
	int* values= new int[ array_size ];

	for (int i=0 ; i < array_size; i++)
		values[i]=i;

	if((FD = gfal_open ( filename, O_WRONLY|O_CREAT, 0644 )) < 0)
	{ 
		perror ("error in gfal_open");
		exit(1);
	}
   	cout << "File is successfully opened\n";

	if ((rc=gfal_write (FD, values, block_size)) != block_size )
        {
		if (rc < 0) perror("error in gfal_write ");
		else cerr << "gfal_write returns "<< rc <<endl;
	}
   	cout << "File is successfully written\n";

	if ((rc= gfal_close (FD)) < 0)
	{
		perror ("error in gfal_close");
		exit(1);
	}
        
	cout << "Close successful ..."<<endl;
} 

GFAL Java API

GFAL Java API has been converted from the C API, by developing an object-oriented wrapper in Java [1]. Please refer to the latest release for developing applications.

Classes

• GFalDirectory - Remote directory

** GFalDirectory() - constructor

**  void	closeDir()
Close the directory associated with this object.

**  static void	makeDir(java.lang.String dirName, int mode)
 Creates a new directory with permission bits taken from mode.

** void	openDir(java.lang.String dirName)
Open a directory

** java.lang.String[]	readDir()
Read the directory associated with this object.

** static void	rmDir(java.lang.String dirName)
Removes a directory if it is empty.

• GFalFile - Remote file

 ** constants: (all integer) CREAT, LARGEFILE,	READONLY, SEEKCUR, SEEKEND, SEEKSET, TRUNC, WRITEONLY 
 
 ** GFalFile()
 constructor

 ** void	closeFile()
 Closes the file opened by openFile.

 ** void	createFile(java.lang.String name, int mode, boolean isSurl, boolean isLargeFile)
 Creates a file

 ** java.lang.String	getLFN()
 Returns the LFN of the file associated with this object.

 ** java.lang.String	getSurl()
 Returns the SURL of the file associated with this object.

 ** void lfcRegisterFile(java.lang.String logicalFileName)
 Registers the file associated with this object in the lfc catalog.

 ** long lseekFile(long offset, int seekMode, boolean isLargeFile)
 Positions/repositions to offset the file associated with this object.

 ** void openFile(java.lang.String fileName, int flags, int mode, boolean isLargeFile)
 Opens a file according to the value of flags.

 ** byte[]	readFile(int size)
 Reads size bytes from the file'

 ** int	writeFile(byte[] buffer)
 Writes buffer data in the file

• GFalUtils - Useful utilities

 ** java.lang.String cError - C error code'

 ** GFalUtilities() - 'constructor'

• • int accessFile(java.lang.String fileName, int mode)

 Checks the existence or the accessibility of the file/directory path according to the bit pattern in amode using the real user ID.

 ** void	chmodFile(java.lang.String fileName, int mode)
 Change access mode of a file/directory.

 ** void	deleteFile(java.lang.String fileName)
 Remove a file entry.'

 ** long[]	lstatFile(java.lang.String fileName, boolean isLargeFile)
 Gets information about a file or directory.

 ** void	renameFile(java.lang.String oldName, java.lang.String newName)
 Rename a file or a directory.

 ** long[]	statFile(java.lang.String fileName, boolean isLargeFile)
 Gets information about a file or directory.

Exceptions

* GFalDirectoryException
* GFalFileException	 
* GFalUtilitiesException

Example

import it.infn.catania.gfal.*;


public class testGfal 
{

	public static void main (String args[])
	{

		String msg = "Usage:\n\testGfal <SE_hostname> <LFN>";
		msg += "\n\te.g. testGfal aliserv6.ct.infn.it lfn:/grid/gilda/scardaci/testGFal.dat";

		if (args.length != 2)
			System.exit(1);
		
		String fileName = args[0];
		String lfcName = args[1];

		try {
			GFalFile gFalFile= new GFalFile();
			gFalFile.createFile(fileName,644,false,false);

			byte[] dati = new byte[1024];
			for(int i=0;i<dati.length;i++)
				dati[i] = (byte)i;

			int ret = gFalFile.writeFile(dati);
			if (ret==-1)  
				System.exit(1);   
			gFalFile.closeFile();

			gFalFile.lfcRegisterFile(lfcName);

			gFalFile.openFile(gFalFile.getLFN(),GFalFile.READONLY,644,false);
			byte[] dati1 = gFalFile.readFile(1024);

			for(int i=0;i<dati1.length;i++)
				System.out.println("dati1["+i+"]="+dati1[i]);		

			gFalFile.closeFile();

		} 
		catch (GFalFileException exc) {
			exc.printStackTrace();
		}

	}

}

External Links

GFAL man pages provides more information about library functions