Data Management Web Portal - User Guide
From EGEE-see WIki
This Wiki page is a part of SEE-GRID Gridification Guide. It is contributed by Belgrade University Computer Centre.
The described software can be downloaded from Data Management Web Portal - Configuration and Integration with Applications.
Introduction
The Grid Data Management Web Portal enables easy interaction with grid file catalog using simple Web-based user interface. It was developed and tested on LCG-2 middleware. The portal provides the following grid data management functions: directory browsing, directory management operations (creating new subdirectories, renaming/moving and deleting directories, replicating directory content, reading directory attributes and permissions), copying and registering of new files to the catalog, file deletion, reading file attributes and permissions, replica management operations (download, replication and deleting) and alias management operations (creating, deleting, renaming/moving). The Web portal’s appearance and the set of operations offered to the user can be controlled using a configuration file. The portal offers its users the ability to manipulate the data on the grid without detailed knowledge about the underlying infrastructure and grid middleware. Its user interface provides an easy and fast way of performing data management operations both to the people who are using the grid for some specific usage related to their area of expertise and for the software developers and administrators, helping them to perform necessary data management operations faster and focus on other aspects of their work.
Developers who want to integrate this tool with their application must be aware that its current implementation performs grid operations on behalf of a single grid user, due to the limitations of GSI used in gLite 3 and LCG, as described in the section 4.5.5.3.2. An implementation of a workaround for this is planned, as well as integration with FTS and AMGA services.
System usage
Four key pages of the Web user interface are Data Management page, File Details page, Directory Details page and Upload page. Besides this page there are additional pages used for logging on to the system, choosing target directories for various directory operations and pages used for showing various reports.
When a user initially accesses the portal, the Login page requesting authentication is displayed. After providing correct user name and password user is redirected to the Data Management page. The user’s identity is kept during the duration of web session
The Data Management page is used for browsing the grid catalog, selecting files and performing actions over currently chosen directory. This page consists of the navigation menu in the upper left corner of the screen, and two work areas. Area on the left contains a list of the actions that could be performed on the current directory, and the area on the right contains the tree pane, used for displaying grid catalog structure and directory browsing, and the current folder contents, where files and subdirectories within the current directory are listed. Data Management page is shown on Figure 1.
The same navigation menu as in the left corner of the Data Management page is shown on all pages. It contains links Data Management and Upload for quick access to respective pages, and Info link leading to the page with information about the development of the portal, its authors and usage. An additional link can be added by setting its title and link properties in portal configuration.
From the Data Management page users can perform various actions on the current directory. As shown in the Current directory actions list users can create new sub directory, rename or move the current directory, delete it, replicate files contained in it, or add links to other directories. If the user tries to execute an operation for which he/she doesn’t have adequate permissions, a report informing the user is shown below the title.
When creating new sub directory or renaming current directory a pop-up window appears, prompting the user for the name of new subdirectory or the new name of the current directory. After this, a new page, called Folder Chooser, is shown. Folder Chooser page is shown in Figure 2. It enables user to browse between existing directory structure in the LFC and choose new parent folder for directory that is being renamed/moved. Simple renaming of a directory is done by choosing old parent directory as the new parent directory. In that way only the name of the directory will be changed and the path will remain the same. Moving of a directory is done by typing old name of the directory in the prompt box and choosing a directory different from the old parent directory as new parent directory.
Because actions of deleting current directory content and replicating the current directory can be applied to the files contained in the current directory as well as to all files contained in the current directory and its sub directories, there are two links for each of the above mentioned operations. Option Delete whole directory deletes directory content as well as the directory itself from the LFC. Before replicating the content of the directory users can choose a Storage Element for new replicas from the drop-down list. Replicas will be created for all contained files that don’t already have a replica on a chosen Storage Element.
Users of the application can easily create new aliases (links) for current directory or add links to other directories to current directory contents. When a user chooses Add link to a directory option a Folder Chooser page is shown. It enables user to find the target directory for new alias. After choosing a target directory, Data Management page is reloaded with new alias listed among other files and links contained in the current directory. Create link to current directory is another option for creating new aliases but this option uses current directory from Data Management page as a target directory for new alias and on Folder Chooser page user chooses a parent directory that will contain the new alias.
Directory tree part of the Data Management page shows the path to current directory and enables user to browse trough the directory structure. Content of the current directory is shown in the rightmost area of the Data Management page. In this area subdirectories of current directory are listed along with files and links contained in current directory. All aliases are hyperlinked to pages showing the contents of target directories or file details in case target is a file. Creating new file aliases (links) is explained further down. If a target file or directory is renamed or moved the existing links will be shown as broken and they will no longer be hyperlinked to its target. Each item in LFC can have a comment assigned to it. The length of the part of the comment that will be shown on Data Management page is determined in configuration file. Long comments are shown entirely as tool tip text when user positions a mouse pointer over the comment. Changing of assigned comments is done on File Details page for files and on Directory details page for directories.
File Details page is shown when a user clicks on a file contained in a current directory or on an alias for a file that is actually contained by another directory. File Details page shows all information available about the chosen file (file attributes and permissions) and the actions that can be performed on the file. File Details page is shown on Figure 3.
The actions that can be performed on a logical file are changing the comment assigned to the file, download (which opens the dialogue for saving or opening the file in user’s Web browser), deletion of all file replicas, renaming or moving the file, deleting an existing file alias, creating new file alias, renaming and moving existing file aliases and file replication to a selected SE. The actions that can be performed on single replica of a file are downloading to the client computer and replica deletion. When a user chooses to download a file and not a specific replica, the SE from which the download will be executed is chosen by the grid. When creating new file alias, similar to the case of creating an alias for a directory, users will be redirected to Folder Chooser page to choose a parent folder for new file alias. Delete alias option deletes an alias selected in the drop-down list that contains all the aliases for current file. As in case of renaming/moving directories, after choosing rename/move option for an alias or for the file itself, users should enter new name in a pop-up window and the choose a new parent at the Folder Chooser page. If a logical file is renamed or moved all aliases existing for the file before renaming/moving are declared broken and not longer listed in the aliases list.
Directory Details page shows all the information contained in the LFC about the current directory. That information is permissions, date information, information about the owner and the comment. Bottom table on the Directory Details page contains statistics about the contained items in the directory and a link to Alias Management page. Alias Management page is used for managing aliases already contained in the directory or for adding new aliases. From this page users can delete broken and non broken aliases, rename aliases or add new ones. Directory Details page is shown on Figure 4.
The Upload page is used to copy files from the client’s local file system to the grid and register them in the LFC catalog. The page contains the upload form for selection of file to be uploaded, destination SE where a replica is to be created, and the logical grid directory and file name to be used when registering the file. If the portal is integrated into some other web application in top left corner of the screen link Multiple Upload is shown. In order to enable parallel upload of multiple files new Upload page opens in a new window each time link Upload on the Data Management page or link Multiple Upload on the Upload page is clicked. The uploading and registering of a file to the grid has two phases: uploading of the file from the client computer to a temporary location on the UI, and copying and registering of the temporary UI file to the grid. During the upload a pop-up with progress bar is shown. If the upload process has completed without any errors a report is shown to the user. From that report page users can view basic details of uploaded file, go to the File Details page of the uploaded file, return to the upload form or return to the Data Management page. Pages Upload and Upload Report are shown on Figure 5 and Figure 6 respectively. After closing the window with upload report users should refresh the Data Management page so the list with contained files would be up to date.
The upload functionality supports optional additional file processing after upload. The user can use the portal configuration to specify if that the file on UI should be processed by a custom script or executable program. If the processing ends successfully, the resulting file will be sent to grid instead of original one. If the processor exits with an error, the content of its stdout and stderr streams and executed command are shown to user through the Web interface so that he/she can find the cause of the error.
Conclusion
The only thing a client computer needs to use the features offered by Grid Data Management Web portal is a JavaScript enabled Web browser. The client/server communication is implemented via HTTP protocol. Access is possible from any computer with the internet connection, without requiring installation of any additional software. This is one the most important advantages of the Data Management Web Portal over tools provided by LCG-2 middleware, because provided tools require execution directly from the UI computer using shell commands or low-level grid APIs.
The possibility of configuring the appearance of the Web portal, choosing the set of data management operations offered to user and requesting data management operation via HTTP interface makes the integration with other Web applications very easy. The concept of the upload processor gives us the possibility to expand basic upload functionality.
