“PROOF on Demand” is a set of utilities, that allows to start a PROOF cluster at user request, on a batch farm or on the Grid. It provides a plug-in based system, which allows to use different job submission frontends, such as LSF or gLite WMS. Main components of “PROOF on demand” are the PROOFAgent and the PAConsole. PROOFAgent provides the communication layer between the xrootd redirector/PROOF master on the client machine and the PROOF workers on the batch or Grid machines, possibly behind a firewall. PAConsole provides a user-friendly GUI, and also allows to easily manage PROOF worker job submissions to different systems, which can later function as one uniform cluster. Installation is simple and doesn't require administrator rights, and all the processes run only in user space. “PROOF on Demand” gives users, who don't have a static PROOF cluster at their institute, the possibility to enjoy the full power of interactive analysis with PROOF.
Tuesday, February 3, 2009
gLitePROOF has been upgraded
“PROOF on Demand” is a set of utilities, that allows to start a PROOF cluster at user request, on a batch farm or on the Grid. It provides a plug-in based system, which allows to use different job submission frontends, such as LSF or gLite WMS. Main components of “PROOF on demand” are the PROOFAgent and the PAConsole. PROOFAgent provides the communication layer between the xrootd redirector/PROOF master on the client machine and the PROOF workers on the batch or Grid machines, possibly behind a firewall. PAConsole provides a user-friendly GUI, and also allows to easily manage PROOF worker job submissions to different systems, which can later function as one uniform cluster. Installation is simple and doesn't require administrator rights, and all the processes run only in user space. “PROOF on Demand” gives users, who don't have a static PROOF cluster at their institute, the possibility to enjoy the full power of interactive analysis with PROOF.
Posted by Anar Manafov at 10:47 AM 0 comments
Labels: gLitePROOF, PoD
Tuesday, September 2, 2008
gLitePROOF 2.0.6
The gLitePROOF v.2.0.6 is out.
This release should make gLitePROOF a lot more user-friendly, it contains many new features and bug fixes. This release was tested against major HEPCG gLite Grid sites, such as FZK, DESY, UNI-Hannover, Wupertal and more.
I am very sorry for the delay of this version, but additional testtings were required.
Download
User's manual
Project's Trac (Wiki, Tickets, Source Browser)
Release Notes:
gLitePROOF
Release 2.0.6 2008-09-02
- this version is based on ROOT 5.18 (see: workers are dropped if ROOT 5.20 or ROOT 5.21.02 are used - [Trac Ticket #72] )
- implemented a part of [Trac Ticket #53].
Since r1963 there is a possibility to use xrootd pools on WNs.
When users can't use external xrootd clusters or any other data source,
they may want to use local WNs as pools. In this case gLitePROOF makes UI
as a redirector and all connected workers as servers. And one can use xrdcp
to distribute data files on the cluster. Example:
xrdcp dstarmb0.root root://depc218.gsi.de:20094//tmp/
Where depc218.gsi.de - is a gLitePROOF UI and 20094 is a port for xrootd (1094 - is a default one).
In the current implementation there is only one major restriction, that a port,
which the user has assigned for xrootd should be opened on WNs.
In the next version of PROOFAgent there will a proxy provided,
so that this restriction will be voided.
- documentation has been updated.
- minor bugs have been fixed.
- Added an ability to download a PROOF log files from worker nodes. [Trac Ticket #77]
- using GAW v.3.2.0.1974, PROOFAgent v. 1.0.5.1977 and PAConsole 1.0.5 (see apps' ReleaseNotes files for more details).
glite-api-wrapper
Release 3.2.0 2008-08-20
- The project has been moved to Eclipse 3.4 "Ganymede"
- Fixed typos.
- Fixed the build of unit tests. There was a hard coded path to XERCES-C.
- Added JobLogInfo methods to the CJobManager class. Actually there are two JobLogInfo methods.
* The first one returns a container of LB Events of a given job.
* The second one formats LB Events and streams this information to a given ostream object.
- MiscCommon: added a new function-helper. It helps to retrieve user's home directory by the name of a user.
- MiscCommon: the smart_path function has been improved to handle such a case like:
~user_name/../../ (see its documentation for more information)
PROOFAgent
Release 1.0.5 2008-08-20
- Fixed typos.
- The project has been moved to Eclipse 3.4 "Ganymede".
- Fixed an improper behaviour when PROOFAgent workers fail to connect to XROOTD on Grid WNs [Trac Ticket #76].
PAConsole
Release 1.0.5 2008-08-30
- The project has been moved to Eclipse 3.4 "Ganymede".
- Fixed many typos.
- Show message box if PAConsole user submits jobs but server is not running [Trac Ticket #70].
- Added a possibility to retrieve jobs logging information from the gLite LB.
Now using a context menu of the Grid jobs list one can get a full log of the selected job. [Track Ticket #65]
- It is now possible to submit (with different jdl files) and monitor several jobs simultaneously.
Parametric jobs (gLitePROOF default jdl) monitored as well.
Every time user clicks the submit button a new job will be submitted and added to the monitoring.
In this case users can add more workers to a gLitePROOF cluster on the fly and keep tracking all of them.
Users also can remove job from monitoring using context menu of the jobs list control.
Remove is not supported for children jobs, only single jobs or parents can be removed from the monitoring. [Track Ticket #71]
- Optimizations in jobs' monitoring, which led to reductions of calls to gLite LB.
- Added a Preferences page. Users now are able to adjust PAConsole settings. Currently the following settings are available:
* Job Status update interval
* Workers update interval
more will come... [Track Ticket #73].
- PAConsole wasn't using the defined jdl file, if a user changed jdl file name manually (without Browse button).
- GUI fixes.
- Minor bugs fixing.
+ many minor and major BUGs have been fixed in all modules (see Project's Trac for more details).
The date of the next version release will be announced a bit later.
Posted by Anar Manafov at 4:07 PM 0 comments
Friday, June 13, 2008
gLitePROOF 2.0.5
I am pleased to announce the release of gLitePROOF version 2.0.5.
Download
User's manual
Project's Trac (Wiki, Tickets, Source Browser)
Release Notes:
gLitePROOF
Release 2.0.5 2008-06-13
- gLitePROOF.sh - a grid job script, its error and signal handling has been improved [Trac Ticket #52].
- added the latest PAConsole.
- documentation has been updated.
- using GAW v.3.1.0.1880, PROOFAgent v. 1.0.4.1883 and PAConsole 1.0.4 (see apps' ReleaseNotes files for more details).
glite-api-wrapper
Release 3.1.0 2008-06-13
- using explicitly libclassad_ns.so(a) instead of libclassad.so(a). This is a ClassAd with namespaces
(see: https://savannah.cern.ch/bugs/?22696)
- ported to gcc 4.3 and gLiteUI 3.1.13
- minimum version of Autoconf that can successfully compile GAW's configure.ac is 2.61 now
- added the BOOST serialization library
- Handling WMP exceptions while delegating credential [Trac Ticket #68]
PROOFAgent
Release 1.0.4 2008-06-13
- documentation has been updated
- minor fixes
PAConsole
Release 1.0.4 2008-06-13
- Added a context menu item (Grid page), using which one can retrieve job
output files of the selected job [Trac Ticket #48].
- Now PAConsole supports serialization [Trac Ticket #47].
It remembers users settings and, for example, the last submitted job saved too.
Which allows user to restart PAConsole and still monitor the last job.
Configuration data saved to and loaded from $GLITE_PROOF_LOCATION/etc/proofagent.cfg.xml.
- ReleaseNotes should be a part of distr [Trac Ticket #61].
+ many minor and major BUGs have been fixed in all modules (see Project's Trac for more details).
New release of the stable gLitePROOF planned for 15.08.2008.
Posted by Anar Manafov at 4:28 PM 0 comments
Labels: gLitePROOF
Thursday, June 5, 2008
PAConsole serialization
Starting from gLitePROOF 2.0.5.1858 PAConsole supports serialization.
What does it mean for the users?
Exactly the following. Now PAConsole "remembers" all basic settings and next time user starts PAConsole, it will load previously saved settings. Main advantage is that users finally are able to monitor the last submitted job even after they restart PAConsole.
New release of the stable gLitePROOF planned for 15.06.2008.
Posted by Anar Manafov at 5:41 PM 0 comments
Labels: gLitePROOF, PAConsole, serialization
Wednesday, February 27, 2008
gLitePROOF user's manual
The gLitePROOF project has just gotten its user's manual. Please meet it at the Documentation Portal of the project.
Posted by Anar Manafov at 12:07 PM 0 comments
Monday, February 4, 2008
RGLite plug-in in ROOT
Now RGLite plug-in is officially a part of ROOT. You can find the latest version of the plug-in in ROOT's trunk.
Posted by Anar Manafov at 9:16 PM 0 comments
Monday, January 28, 2008
glite-api-wrapper v3
Yesterday I have released GAW v.3.
#1 Changing LFC current directory to "/grid/dech"
CGLiteAPIWrapper::Instance().Init();#2 or retrieving the current LFC directory
CCatalogManager * pCatalog( &CGLiteAPIWrapper::Instance().GetCatalogManager() );
gaw_lfc_cwd cwd;
cwd.m_dir = "/grid/dech";
pCatalog->Run( cwd );
string sRetVal;#3 or list current LFC directory to a container
gaw_lfc_pwd pwd;
pCatalog->Run( pwd, &sRetVal );
cout << "---> Current catalog directory: " << sRetVal << endl;
LFCHelper::LFCFileInfoVector_t file_list;or to the stdout
gaw_lfc_ls ls;
ls.m_dir = "/grid/dech";
pCatalog->Run( ls, &file_list );
gaw_lfc_sls ls;more examples one can fined here
ls.m_dir = "/grid/dgtest";
cout << "---> Listing directory: " << ls.m_dir << endl;
pCatalog->Run( ls, &cout );
Additionally the version includes a number of cosmetic changes and an updated source documentation.
supported features
- Workload Management System operations:
- job submission - normal, DAG and parametric jobs (gLite WMProxy API),
- smart look-up algorithm for WMP-Endpoints,
- job status querying (gLite LB API),
- job output retrieving (Globus GridFTP).
- File Catalog operations (gLite/LCG LFC API):
- smart session manager,
- set/query the current working catalog directory,
- list files, directories and their stats,
- add/remove files in a catalog namespace,
- add/remove directories,
- add/remove replicas from a given file.
- smart session manager,
- An executive logging.
- Support of an external xml configuration file with according XML schema.
Posted by Anar Manafov at 6:09 PM 0 comments
Labels: gLite, glite-api-wrapper, LFC
Friday, January 11, 2008
ROOT HTML documentation for RGLite plug-in
I've just finished the ROOT style HTML documentation for RGLite plug-in.
Only one comment: it takes really a lot more time to write, compile and support a good documentation with the ROOT Automatic HTML Documentation style than with doxygen, for example.
Now I love doxygen much more than I used to.
Anyway, the first draft of the RGLite documentation is ready.
Posted by Anar Manafov at 8:14 PM 0 comments
Labels: documentation, RGLite
Tuesday, January 8, 2008
New XML schema for the GAW configuration
glite-api-wrapper (GAW) has just gotten a new XML Schema for the configuration file, starting from r1630. Documentation pages could be found here (a XML Schema documentation).
BTW, from now on we use a Xerces-C DOMBuilder interface to validate and parse GAW configurations.
DOMBuilder is an interface introduced by the W3C DOM Level 3.0 Abstract Schemas and Load and Save Specification. DOMBuilder provides the "Load" interface for parsing XML documents and building the corresponding DOM document tree from various input sources.
Posted by Anar Manafov at 7:32 PM 0 comments
Labels: gLite, glite-api-wrapper
Monday, December 31, 2007
ClassAds and gLite UI 3.1(3.0)
There is a long-living problem in gLiteUI 3.x. I decided to post this topic here since I have received many requests of help in this issue (I received them most probably because I was the first who posted the according bug #22696 in CERN Savannah).
The problem is that one needs to compile ClassAds with namespace support enabled, because gLite UI distributive contains ClassAds which compiled without support of namespaces, though some of gLite API libraries (WMSUI for example) require ClassaAds with namespace support. And if one tries to compile its application, which uses gLite WMS API it will end up with the something like:
libglite_wms_util.so.0: undefined reference to `classad::ClassAdParser::ParseClassAd(std::basic_istream >&, bool)'
libglite_wms_jdl.so.0: undefined reference to `classad::ExprList::insert(__gnu_cxx::__normal_iterator > >, classad::ExprTree)'
libglite_wms_jdl.so.0: undefined reference to `classad::ExprList::ExprList[in-charge](std::vector > const&)' libglite_wms_jdl.so.0: undefined reference to `classad::ClassAd::SameAs(classad::ExprTree const) const' libglite_wms_jdl.so.0: undefined reference to `classad::ExprList::erase(__gnu_cxx::__normal_iterator > >, __gnu_cxx::__normal_iterator > >)'
libglite_wms_jdl.so.0: undefined reference to `classad::ExprList::erase(__gnu_cxx::__normal_iterator > >)'
libglite_wms_jdl.so.0: undefined reference to `classad::ExprList::push_back(classad::ExprTree)'
libglite_wms_jdl.so.0: undefined reference to `classad::ClassAd::EvaluateExpr(classad::ExprTree const, classad::Value&) const'
libglite_wms_jdl.so.0: undefined reference to `classad::ClassAd::find(std::basic_string, std::allocator &
Well, I didn't so far find a better way, but just compile own version of ClassAds library.
Download classads-0.9.6 from here.
tar -xzvf classads_0.9.6.tar.gzBTW, the following patch helps to compile CLASSADS with gcc-4.x (for classads-0.9.8) or you can use classads-0.9.9, where the problem is FIXED already.
cd classads_0.9.6
./configure --enable-namespace
make
make install
--- ../classads-0.9.8.orig/classad_stl.h 2006-06-14 21:39:30.000000000 +0200
+++ classad_stl.h 2006-11-09 10:59:25.000000000 +0100
@@ -48,7 +48,7 @@
#define classad_hash_map std::hash_map
#define classad_slist std::slist
#elif defined(__GNUC__)
- #if (__GNUC__ == 3 && __GNUC_MINOR__ > 0)
+ #if ((__GNUC__ == 3 && __GNUC_MINOR__ > 0) || (__GNUC__ > 3))
#define classad_hash_map __gnu_cxx::hash_map
#define classad_slist __gnu_cxx::slist
#else
--- ../classads-0.9.8.orig/Makefile 2006-06-14 21:39:30.000000000 +0200
+++ Makefile 2006-11-10 08:19:13.000000000 +0100
@@ -76,15 +76,17 @@
shared: shared.C
$(CPlusPlus) -fPIC -shared -o libshared.so shared.C
+test: classad_version classad_unit_tester classad_functional_tester test_xml extra_tests
+ ./classad_version
+ ./classad_unit_tester -all
+ ./test_xml
+ ./extra_tests
+
install:: $(LIB_NAME) classad_functional_tester classad_unit_tester classad_version
- mkdir -p $(LIB_INSTALL)
- mkdir -p $(BIN_INSTALL)
- mkdir -p $(INCLUDE_INSTALL)
- cp $(LIB_NAME) $(LIB_INSTALL)
- cp classad_functional_tester $(BIN_INSTALL)
- cp classad_unit_tester $(BIN_INSTALL)
- cp classad_version $(BIN_INSTALL)
- cp $(HEADER_FILES) $(INCLUDE_INSTALL)
+ mkdir -p $(LIB_INSTALL) $(BIN_INSTALL) $(INCLUDE_INSTALL)
+ install -m 644 $(LIB_NAME) $(LIB_INSTALL)
+ install -m 755 classad_functional_tester classad_unit_tester classad_version $(BIN_INSTALL)
+ install -m 644 $(HEADER_FILES) $(INCLUDE_INSTALL)
clean: tidy
rm -f classad_functional_tester classad_unit_tester classad_version