Data Management Web Portal - Configuration and Integration with Applications

From EGEE-see WIki

This Wiki page is a part of SEE-GRID Gridification Guide. It is contributed by Belgrade University Computer Centre.

The usage of this software is described in Data Management Web Portal - User Guide.

1 Introduction
2 Configuration file format
3 URL query string parameters
4 Adjustable features
5 Downloads

Introduction

Data management is a necessary part of any system that utilizes grid infrastructure for either commercial or scientific purposes. Grid Data Management Web Portal can be integrated into a wider application that would offer a various set of services to the users. The appearance and the functionality of the web portal are defined by the configuration parameters that are read from a configuration file. Table 1 gives an overview of all the configuration parameters used in Grid Data Management Web Portal.

Application access control parameters
Parameter name	Default Value	Required	Purpose
AccessControlEnabled	1	no	If this parameter has a value of 1, parameters Username and password are compared to the username and password entered by user. If this parameter has a value of 0, there is no authentication.
Username	null	no	Value of this parameter is used as username for user authentication.
Password	null	no	Value of this parameter is the password for user authentication.

User operations access control
EnableFolders	1	no	If this parameter has a value of 1, users are allowed to browse through the directory structure and initiate operations over directories. If this parameter has inactive value, users can only view the content of current directory.
EnableUpload	1	no	This parameter determines if copying of files from local computers to the UI and the registration of those files in the LFC is enabled.
EnableDownload	1	no	Value 1 or 0 determines whether download of the files is allowed or not.
ReadOnly	0	no	Value 1 for this parameter means that the users can not alter the content of the directories and initialize operations over files and directories. If the value is 0, users have full access. Browsing is enabled for both values.

Grid environment parameters
VO		yes	Name of the Virtual Organisation the user belongs to.
RepManagerRoot		yes	Path of the grid directory that will be considered as root directory.
bdii.host.name	null	no	The name of BDDI host that publishes the list of available Storage Elements.
bdii.host.port	null	no	Port number used for access on the host defined by bdii.host.name.
SE.ignore.list	null	no	List of Storage Elements to be ignored even if published by BDII

Directories on UI for temporary files
UploadDirUnprocessedFiles		yes	Defines the directory on UI computer that will be used for storing a temporary file during upload.
DownloadDirUnprocessedFiles		yes	Defines the directory on UI computer that will be used for storing a temporary file during download.

Appearance settings
UpLevelLinkName	null	no	Name of the additional navigational menu item.
UpLevelLinkURL	null	no	URL for additional navigational menu item.
InfoLinkURL	null	no	URL of the page used as Info page.
PageFooterHtml	null	no	Adaptable HTML code that will be shown in bottom of every page.
CommentColumnLength	null	no	Determines the length of the comment part that will be shown on Data Management page. If this parameter is not set or has a value of 0, comments will not be shown on Data Management page.

Pre-processing of files at upload
UploadProcessor	null	no	Specifies a program on UI that will be used to process the file after the upload to UI and before the registration in the LFC. In order to pre-process the file parameter UploadDirProcessedFiles should also be set.
UploadDirProcessedFiles	null	no	Defines a directory on UI that will be used for storing a temporary, processed file.
UploadProcessorExtension	null	no	Expected extension of a processed file. If not provided an extension identical to one before processing is expected.
GenerateUploadProcessorFilenames	null	no	If this parameter has a value of 1, temporary names based on system time will be used. If the parameter has a value of 0, original names will be used.
UploadProcessorTimeout	null	no	Defines the maximum amount of time in seconds allowed for processing. If the processing does not end until the time expires, it is considered an error has occurred and the upload is aborted.
UploadRegistrationExtension	null	no	Extension that will be concatenated to the file name before the registration.

Table 1: Overview of configuration parameters

[edit]

Configuration file format

Configuration files that are used in this application are written in a format defined for Java properties files. Java properties files are written according to the following rules:

Files are processed one line after another, and a line ends with an end-of-line character (\n\r, \n, \r) or by end-of-file character.
Comment lines start with '#' or '!' There are only single line comments.
Lines that are not comments contain pairs of parameter name and parameter value. The name and the value of the parameter are divided by '=' or by ':' characters or by any white character different from end-of-line and end-of-file characters.
First non-white character after the name and the separation character is considered the start of the parameter value part. End of the parameter value part is the end of the line it is in. The value of the parameter comprehends all the characters except not commented '=', ', end-of-line, and end-of-file characters.
The value of the parameter is continued onto the next line if a '\' character is put in front of end-of-line character. The value continues at first non white character other than end-of-line and end-of-file.
The files are written with Latin1 character encoding (ISO 8859-1). Unicode and special characters can be given in format used for Java strings and characters (like '\n' for new line and '\u000C' for unicode characters).

Table 2 shows an example how a properties file could look like with all the parameters set with an example value.

#========================
# Data Storage properties
#========================
# Virtual Organisation (required)
VO = seegrid
#
# Name of the BDII host that publishes the list of avaialble  SEs<br> 
bdii.host.name = bdii.phy.bg.ac.yu
#
# Port number on BDII host
bdii.host.port = 2170
# list of SEs that are to be ignored even if published by BDII
# The SEs are separated by space character (optional parameter)
SE.ignore.list = silyon02.cc.metu.edu.tr yesim.grid.boun.edu.tr
#
# Application's root folder (required)<br> RepManagerRoot=/grid/seegrid/VIVE 
#======================================
# Application access control properties<br> #======================================
# 1 to use application access control, 0 to disable application access control<br> AccessControlEnabled = 1
#
# Username used for login
Username = someuser
#
# Password used for login
Password = somepassword
#=========================
# User actions permissions
#=========================
# Enable folder browse (0/1)
EnableFolders=1
#
# Enable upload action (0/1)
EnableUpload=1
#
# Enable download action (0/1)
EnableDownload=1
#
# Enable only read only actions (0/1)
ReadOnly=0
#
#=====================
# UI directories setup
#=====================
#
# UI temp. directory for unprocessed uploaded files (required) <br>UploadDirUnprocessedFiles = /home/oki/work/upload/unprocessed
#
# UI temp. directory for unprocessed downloaded files (required) <br>DownloadDirUnprocessedFiles = /home/oki/work/download
#
#=============================
# GUI customisation properties
#=============================
#
# Name of custom menu item
UpLevelLinkName = Working Set
#
# Destination URL for custom menu item
UpLevelLinkURL = /BVis/SceneGenerator.exe?a=DIR
#
# Custom page footer
PageFooterHtml = <p><center>RCUB, 2005.</center></p><br>#
# Custom Info page
InfoLinkURL = info2.htm<br>#
#============================
# Upload processor properties
#============================
#
# Upload processor executable pathname
UploadProcessor = /usr/local/bin/FieldServer2
#
# UI temp. directory for processed uploaded files (required)<br> UploadDirProcessedFiles = /home/oki/work/upload/processed
# Expected extension for processed file("ext" or ".ext" to specify, empty string or . for no extension)
UploadProcessorExtension = CSFE
#
# Enabe/disable (1/0) generation of temporary filenames when uploading
GenerateUploadProcessorFilenames = 1
#
# Maximum interval in seconds for upload processor to finish execution (processing)
UploadProcessorTimeout = 15
#
# End of configuration file

Table 2: Properties file example

[edit]

URL query string parameters

Whenever a request for Data Management, Directory Details, File Details or Upload page is made the parameters of URL query string are checked. If the Grid Data Management Web Portal is a part of a wider unit, setting the value of p, u and ro parameters can change the configuration of the application. These parameters have the following meaning:

Parameter p of the URL query string determines the name of a configuration file from which the configuration parameters will be read. The name of the configuration file is determined as "repmngr"+<p>+".properties", where <p> is the value of p parameter. Default value is null.
Parameter u is used to restrict user's navigation to specific directory and its subdirectories. Users are restricted to a directory defined with <RepManagerRoot>/< u> path, where <RepManagerRoot> is the value of RepManagerRoot configuration parameter explained in Table 1, and < u> is the value of u parameter. Default value is null.
Parameter ro is used to disable users to initiate operations over grid items and change the content of grid files and directories. If set, this parameter overrides ReadOnly configuration parameter. If the value of ro parameter is set to 1, the application is in read-only mode and the users will only be allowed to browse through the directories on the grid and view information about directories and files. If the value of this parameter is 0 users will have full access to all available operations over directories and files. Default value is null.

In order to prevent direct changes of URL query string parameters from the browsers address bar, p, u and ro parameters are protected by MD5<ref>Message Digest 5 algorithm</ref> signatures. Before adding links towards one of the Data Management, Directory Details, File Details and Upload pages, it is first necessary to calculate the signature values for desired values of p, u and ro parameters, set the calculated signatures as the values of HTTP session attributes named propertiesFileSignature, userSubPathSignature and readOnlySignature, respectively for p, u and ro, and then generate links to desired pages with the values of p, u and ro parameters included into the URL query string. If the value of the p, u and ro parameters should be null, then the values for propertiesFileSignature, userSubPathSignature and readOnlySignature session attributes should not be set. For example, it would be possible to determine the values of p, u and ro parameters depending on the users identity after logging on, calculate MD5 signatures, set signatures as the values of propertiesFileSignature, userSubPathSignature and readOnlySignature session attributes and create all links towards Data Management, Directory Details, File Details and Upload pages with values of p, u and ro parameters included into the URL query string.

[edit]

Adjustable features

By setting the value of certain configuration parameters, or the URL query string parameters, several aspects of the web portal can be changed or adapted for the purposes of integrating the portal into a wider system. There is a possibility of reading configuration parameters from a specific file, restricting user's navigation to a specific directory and its subdirectories, adding a link to the navigational menu, changing the looks of the portal, setting an Info page and adding custom HTML code at the bottom of every page. In order to read configuration parameters from a specific configuration file, other than the default file, the value of p parameter should be set in a request. Users are restricted to specific directories and its subdirectories by setting the u parameter.

Additional link in the navigational menu, which is shown in top right corner of some pages, is defined by UpLevelLinkName and UpLevelLinkURL parameters described in Table 1. If the application is integrated into a bigger system, this link could be used to provide a return link to a page from which one of Grid Data Management Web Portal pages was accessed.

The page that will be used as Info page is defined by InfoLinkURL configuration parameter. A link towards this page is a part of navigational menu. Info page could be used for providing information about the version and authors of the system or any other useful information.

The appearance of the application is determined by styles defined in a cascading style sheet (css) file called agenda.css. By redefining the styles the look of the application can easily be adapted to a bigger unit.

A custom HTML code can be shown at the bottom of every page. The value of PageFooterHtml configuration parameter is used as HTML code. One of the possible usages of this functionality is the showing of the copyright text.

[edit]