Event Logger Application
From EGEE-see WIki
This Wiki page is a part of SEE-GRID Gridification Guide. It is contributed by Belgrade University Computer Centre.
Contents |
Introduction
Event Logger is an application that allows users and other applications to store various data which can be used at later time for different kinds of analysis. This tool is generic enough to support arbitrary applications and event logging of any kind of discrete, autonomously generated events that may appear in the domains of grid and network management.
Event Logger consists of:
- Service implementing event archive
- Event recording API used by applications, sensors, and monitoring points or agents to put events into Event Logger
- Basic archive querying API suitable for creation of reports
- HTML-based web application for querying of the archive and display of reports
- Generation and display of aggregations
Service Implementing Event Archive
The core of the Event Logger application is implemented as a service that listens to requests received by other users and applications. Structure of the data that represents this events is designed with the goal to make it as generic as possible so that any application could use it but at the same time simple so that application designers can easily integrate it.
The event data structure is represented by:
- Domain - Identifies the domain (there can be one domain per application, or more applications can share the same domain, also one application can use more than one domain) in which the events are stored
- Target type - A type of target for which the events are being generated, generally this represents a subdomain within the domain, for example some function of the application that is generating different kinds of events or some entity, which represents application services, communication links, etc. (e.g. “network.link”). Contains fields:
- name
- description
- createdOn timestamp
- Target - An entity being recorded in an event
- Event type - A type of an event, for example this can represent the result of a function call. Each target type has a limited set of event types it can be associated with. Contains fields:
- name (e.g. “network.link.delay.TCP” or “vectorValue.processed.serverProcessing”)
- valueType (boolean, int, float, ...)
- unit
- description
- createdOn timestamp
- Event - The smallest stand-alone unit of information (e.g. “delay.TCP” or “processed.serverProcessing”). Contains fields (note that certain fields can be used to group events):
- Id
- targetType
- eventType
- timestamp
- source - Common part of various events that may be used for grouping, e.g. IP address, address of a CE or its full queue identifier
- sourceDetail - Additional source description, e.g. <IP>:<port>, <port>, or WN’s IP. This field can be used freely for various purposes
- destination - Common part of various events that may be used for grouping, e.g. consumer’s or client’s IP address
- destinationDetail - Additional destination description, e.g. <IP>:<port> or just <port>. This field can be used freely for various purposes
- valueFloat, valueInt, valueBoolean, valueString - Only one value is set/used, depending on eventType.valueType
- valueQualifier - Additional numerical attribute describing conditions when event was performed; used for filtering, default is 0
- isAggregation - This event is aggregation of events within an interval
- aggregationType - Type of aggregation, e.g. single run (atomic event), aggregations: mean, count, deviation (median, etc)
- aggregationInterval - Duration of aggregation starting from timestamp, 0 for atomic events
Event Recording API
The API used by applications, sensors, and monitoring points or agents to put events into Event Logger should provide service methods for recording float, integer, Boolean and string values, with optional valueQualifier used to provide an additional numerical attribute that may be used in filtering the events. Currently, only the events with float values are supported. The mandatory arguments are targetType, eventType, timestamp, source, and value. Depending on the context, sourceDetail, destination, destinationDetail, and valueQualifier can be omitted or left empty.
Besides the generation of single events, support for group events is implemented as well as an ability to create aggregated data.
Remote access via Java SE client is provided, examples and instructions are available at Generating Events Using J2SE Client
Basic Archive Querying API
This API provides the interface to the following features:
- List available target and event types
- list of available archive domains
- list available TargetType for one domain
- list available EventType for given TargetType from TargetType-EventType pairs
- List available event source addresses
- list available distinct sources for given TargetType.EventType pair and time interval
- list available distinct sourceDetail values for given TargetType-EventType pair and time interval
- list available distinct sourceDetail values for given source, TargetType-EventType pair and time interval
- List available event destination addresses
- list available distinct destinations for given TargetType.EventType pair and time interval
- list available distinct destinationDetail values for given TargetType-EventType pair and time interval
- list available distinct destinationDetail values for given destination, TargetType-EventType pair and time interval
- List available events for given TargetType-EventType pair and time interval, with optional filtering or grouping by source (optionally by sourceDetail), destination (optionally by destinationDetail), and optional filtering by intervals of event value valueQualifier. For each group of listed event values, the mean, count, and deviation should be calculated as well.
- List of event aggregations by using similar filters and groupings as with single events, but with additional selection of aggregation types (as mean, count, deviation, etc.).
HTML-Based Web Application
HTML-based web application for querying of the event log archive and display of reports lets the user iteratively specify the events to be listed and statistically analyzed. Querying is conducted through several phases:
- Domain Selection – This step is optional, since the domain may be hard-coded for given usage
- Target type selection
- Event type and time interval selection
- Selection of filters and grouping
- Display of events
Generation and Display of Aggregations
Event Logger supports aggregation of event data with an optional aggregation interval. The process of data aggregation is fully configurable and the complete business logic of the aggregator is implemented on the server side.
Event Logger Usage & Examples
Here you can find some examples of applications that use Event Logger. Since the data fields provided by Event Logger are very generic, this examples can be helpful for identifying some of the possible usage scenarios. Developers are required to contact Event Logger support to be able to use the application. For tesing purposes, free access to the TestApp domain is available here. See below for contact details where you can find out how to obtain custom application domain and/or server files if you would like to deploy the application on your own application server.
VIVE Application
VIVE application uses Event Logger for performance measurement purposes. In order to analyze VIVE performance in a grid environment on a large scale, with constantly increasing number of grid sites providing server processes, it was decided to use Event Logger as a measurement archive service for saving performance measurement data.
In VIVE domain, Event Loggers data fields have this meanings:
- Event - Type a type of event
- Target - The entity being measured in an event
- Target type - A type of target (e.g. “VIVE.vectorValue”)
- Event name - A concatenation of target type and event type
- Event Type - Various requests defined within VIVE application client-server protocol, described by strings of the form <outcome>.<feature> Possible outcomes are “processed”, “rejectedByServer”, “cancelledByUser”, while the features are “serverProcessing”, “clientBinderCommunication”, “binderServerCommunication
Contact
Milan Potocnik [milan (d) potocnik (a) rcub (d) bg (d) ac (d) yu]
