ErrorLogger Package -- Feature Log ErrorLogger 019: Timezone Format Statement of suggested enhancement 4/9/04 ---------------------------------- version 2.2.6 Liz Sexton Kennedy pointed out that when software goes truly global, it may be advantageous to have the option of printing the time zone as GMT +/- some number of hours. The strftime formatter already used has this capability. It might even be right to provide an option to alwasy use GMT alone. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- formatTime() is done as a static routine in ELoutput. It is not completely clear what the best method of conveying the format choice will be. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- The capability ought to be spec-ed out and implemented, since it is very low risk. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ Low priority; nobody is clamoring for this "yet." - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- ErrorLogger 018: Exit threshold Statement of suggested enhancement 2/24/04 ---------------------------------- version 2.2.4 Liz Sexton Kennedy (for CDF) pointed out an issue which "is causing major operational problems so I believe it is really needed. Long ago (4years) the error logger called exit() when an ELabort was signaled now it calls abort() which generates a core dump". The original request was for a special severity level. M. Fischler suggested instead a second threshold: I can add a method setExitThreshold(s) which says that if you encounter an error with severity at least s, terminate by using exit(s) as opposed to abort(). So you could say (logger is the ELadministrator here): logger->setExitThreshold(ELabort); // never abort(), instead exit() which prevents all aborts since the (default) abort threshold is ELabort also. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- Done in the suggested way, this change is easy and non-obtrusive. The user can get flexibility of certain messages still triggering the core dump: logger->setExitThreshold(ELabort); logger->setAbortThreshold(ELfatal); This is an easy fix to make, because the issue comes up only in ErrorLog.cc. In fact, in ELtsErrorLog.cc this annoyance is not present. If both thresholds are set, here will be the ground rules in deciding whether to abort or to exit: * The exit threshold is by default HIGHEST_SEVERITY, and the abort threshold is by default ABORT. * The highest applicable threshold applies. * In case of a tie, the action will be abort (this is somewhat arbitrary; the framework ought not to create this situation anyway). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- The proposed solution appears adequate to CDF software. Unless somebody objects, we will do it. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ SHould go in around mid-March. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- ErrorLogger 017: Space after integer types Statement of suggested enhancement 6/24/03 ---------------------------------- version 2.2 Mark Fischler noticed that after an integer is streamed to the errlog, unless a hex format is requested, no space is inserted after that integer -- e.g. 5PhysicsCode instead of 5 PhysicsCode. This is annoying and not consistent with other data types. However, in order not to affect existing outputs of test and validation programs, we should keep the default the way it is, and provide an option like errlog.spaceAfterInts(). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- This is an easy fix to make, because the issue comes up only in ErrorLog.cc. In fact, in ELtsErrorLog.cc this annoyance is not present. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- Comments are solicited. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- ErrorLogger 016: Obtaining name of module Statement of suggested enhancement 6/18/03 ---------------------------------- version 2.2 Andrew Hamilton (request forwarded by Liz Sexton) requests a way to query the ELadministrator to get "a string indicating what it thinks the current module name is." - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- This is likely an easy one, although I would guess the right thing to query is the ErrorLogger rather than the adminstrator. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- Since this is a pure enhancement, and is very low-risk, and CDF will be freezing software very soon, I am going to put this in to a release very quickly. At the same time I will be done enhancvement 015, which has elicited no negative comments. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ Assuming no user objections or suggestions to improve this, coding and documentation for these features should go in by June 27. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- Tested (in basicUsage) and checked in 6/23/03. Unfortunately, it was subsequently learned that this feature does not accomplish what the user needed -- learning what module was currently active, with NO access to an established errlog. This could only be accomplished by magic, since nothing in the program, ErrorLogger-related or not, was tracking that information. ErrorLogger 015: Changing/flushing ostream for a destination Statement of suggested enhancement 3/27/03 ---------------------------------- version 2.2 Wolfgang Wagner noted what he thought to be a peculiar behaviour when an ELdestControl is deleted: the ostream attached to the corresponding ELoutput is not closed, and in his situation, the data becomes available only hours later after the job completes. He'd like to have a way to force closing of the file associated with the destination. The way his code works is that when starting a new run, he ties a new ELoutput with a new ofstream to the ELadministator, and lets the old ELdestControl go out of scope. Upon some thought (see technical information), it turns out that what is really wanted is a way to change or affect the ostream associated with an ELoutput. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- Connecting some behavior to the destruction of an ELdestControl, as originally requested, would be very unwise, since one normal mode of operation is to allow these to go out of scope immediately after setting up the ELdestination. Other usage patterns recover or keep the control, for later changes. In the current package, however, changing the ostream is not one of the options. As long as we don't alter the paradigm that The paradigm in this package is that users who supply explicit ostreams always own the ostreams, and are responsible for keeping them valid while the ELoutput is in use. And if a user constructs an ELoutput supplying file name the ELoutput itself takes care of destruction of the created ofstream. Whatever we do here must preserve this paradigm. Long ago, the design decision was made NOT to support disconnecting an existing destination from the ELadministrator; and I see no reason to perturb that decision here. - - - The methods provided would apply to ELoutput (via ELdestControl, of course). Why not to ELstatistics, ELtsErrorLog, or ELcollected? ELstatistics has an os, but it is for use as a one-shot termination stream in case further stats unreported exist at time of closing. The same issues of changing the stream don't apply to that. ELtsErrorLog is a sor of ErrorLog, not a destination, so os does not apply. ELcollected uses such a different mechanism to transmit its output -- and one that is underexplicit user control anyway -- that the notion of changing the ostream has no unique meaning. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- It is proposed that we should provide the following extensions, as methods of ELdestControl which call corresponding methods of ELoutput: changeFile (ostream &) Replace os with the new supplied ostream. If the old os was owned by the ELoutput, it will be deleted. changeFile ("filename") Replace os with a new ofstream, opened using this filename. This os is owned by the ELoutput. flush() Flush the ostream. ostream& ostream() Supply a reference to the current ostream, which the user can use to invoke flush() or close() or whatever. Obviously this is does allow the user to get into trouble, for example, by closing a file which a still-active ostream will later write to. If the old os was owned (supplied by file name) then of course it is deleted first. While we are at it, we ought to document the notion of when a destination owns and does not own its ostream. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ Being done near the end of June. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Further Technical Information ----------------------------- Turns out that the ostream() method is pretty useless (and hence misleading) since any formating methods done to that ostream have no effect on the formatting of the message (which is done at the errlog stage). Obtaining ostream still would allow a user to learn what stream had been opened so as to insert her own content; but that same can be accomplished by constructing the ELoutput from an ofstream in the first place. Since the benefits are so small, and the potential for confusion (with formatting) and traps is large, I am eliminating this proposed method. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- ErrorLogger 014: Time-zone information Statement of suggested enhancement 11/19/02 ---------------------------------- version 2.1.10 Eric Wicklund notes an instance where the date/time on a user's report was hard to comprehend, because the job was done remotely and nobody knew the time zone. He has requested time-zone info be included with timestamps in ErrorLogger outputs. It is suggested that this addtional info be made the default. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- Although this would change the behavior of the log output, it would not change it in any way that would invalidate automated verification (since the timestamp output, if enabled, inherently varies from job to job). The time zone is hard to get in a portable way, but Greenwich Mean Time is not. The suggested output format is, then, 19-Nov-2002 16:38:17 [21:38 GMT] In some cases, the date in GMT will differe from the local date; but given the local date and time and the GMT time, no ambiguity can exist. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- It is proposed that we do this modification when doing the debug-message syntax changes. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ Coding for these two improvements should start in mid-December. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Further Technical Information ----------------------------- This can be coded in a nice, portable way, and we can get the time zone (as opposed to GMT) thus 19-Nov-2002 16:38:17 CST - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done 12/6/02 ---------------- Tested and checked in. Will be present in v2.2 and in next release. ErrorLogger 013: Debug-Message Syntax Statement of suggested enhancement 9/24/02 ---------------------------------- version 2.1.10 Yuyi Guo requests additional error severity levels (see Enh. 012). However, upon looking at the needs, MF and JBIK have noted that what is really missing is a separate form for sending things for debugging, such that this output can easily be disabled, and when disabled the code using it can be skipped with very little runtime penalty. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- Upon reflection, and after listening to BTeV framework discussions where the concept of debugging output that can be disabled came up, MF and JBK have reached the following tentative conclusions: i) It is desirable to have a separate form for sending things for debugging, such that this output can be choked off with very little runtime penalty. ii) The best syntax we could develop for this would be: framework setup -- errlog.setDebugVerbosity (3); // 0 is the default user code -- errlog(2) << "someString" << moreData << endmsg; errlog(4) << "otherString" << endmsg; In this case, the "someString" message would come out because 2 <= 3, but the "otherString" would be QUICKLY skipped because 4 > 3. Such messages would always have id of "DEBUG" and severity of ELinfo, and perhaps, if this is easy to implement, would not have time stamping. Existing code which uses errlog(severity, id) would of course work unchanged. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- Yuyi Guo (the requestor for more severity levels to kludge this sort of thing) has said this scheme would be fine. If no comments are received balking at the scheme, we think it should be implemented. It is conceivable that for best performance we might later wish to have a stripped-down "debugLogger" package. But that is a separate ZOOM discussion. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ Since this is a major addition to capability, we will wait a while for some user comments before implementing it. Plans and Priority ------------------ Turns out BTeV also needs/desires this sort of thing; plus others (e.g. Eric and Liz) think it is a good idea. Technical Information --------------------- This is easier than it looks because the mechanism almost exactly follows the rapid discard mechanism. However, it must be done separately for the thread safe logger. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- Tested and checked in. Will be present in v2.2 and in next release. ErrorLogger 012: More Severity Levels Statement of suggested enhancement 9/19/02 ---------------------------------- version 2.1.10 Yuyi Guo requests additional error severity levels. The reason is a desire to "use it (ErrorLogger) as first tool for some debugging purpose. I started to add debugging messages, but found out that we have used all the non error and non warning severity levels for other types of message. So I wonder if it is easy and quick for you to add one or two levels above the incidental level, such as 'debugging' or 'verbosity'? ... It would be nice for users to set up all the request in one place, such as we do it know in ErrorLoggerManager, by setting severity threshold level." - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- (A bit of history -- when the package was started, there was considerable debate about how many levels to provide. In the end, mf took the most that anybody was suggesting, and DOUBLED that number -- thus all these Error2 and Warning2 names. There was a lot of ribbing for providing so many more levels than anybody would need...) Adding one or two additional levels would involve a fairly good bit of routine editing, but not all that much trouble. However, several things perhaps argue against adding new levels for the purpose described: 1) There are workarounds for this purpose which may well actually be superior to the idea of using severity levels for this purpose. Some will be presented below. 2) The ErrorLogger documentation presents **EVERY** option available. Because of this, the bulk of the documentation becomes relevant. Adding a lot more severity levels would be bad from this viewpoint. (Of course, we could provide a "hidden" feature of extra undocumented levels; somehow that seems like a bad idea.) 3) We did not design the system to allow for user-customized sets of levels; perhaps that was a design blunder. But given that the levels are fixed by the ErrorLogger package, it is highly undesirable to treat this set as completely open-ended. 4) It is (barely) conceivable that some user has stored an ELseverityLevel in a persistant object. If we add new levels, this will CHANGE the apparent level of the stored object. (For example, if we add 2 levels, a stored ELseverity object that used to be ELerror will now look like ELwarning.) While we don't think anybody is doing this, it would be unfortunate to foul up an unsuspecting user. Still, if none of the below workarounds are satisfactory, we can add a couple of levels. The possible workarounds are: a) You can defined all debug messages to use a message id of "debug", and set up another destination, which I will call debuglog. Then all the other destinations can set limits of 0 on messages with id = "debug", while debuglog sets no limit on those but may set limits on other sorts of messages. b) In a pinch, there is one extra level available: ELunspecified. But this might be in the wrong place for you to use it the way you want (it is more severe than Error2). Also, although we disparage use of ELzeroSeverity except in setting up thresholds, in fact if you issue a message with ELzeroSeverity that will work fine. A further suggestion, whether or not we add extra severity levels: When setting up for debugging through the ErrorLogger, you might want to set and use up a separate ErrorLog "debuglog" just the way errlog is set up and used. Then you can: * Trivially alter which modules' debug info will be filtered out, using the filter by module capabilities, which are on a logger basis; * Minimize the time cost of filtered-out debug messages when no debugging is going on, by doing debuglog.setDiscardThreshold(ELhighestSeverity). - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- Upon reflection, and after listening to BTeV framework discussions where the concept of debugging output that can be disabled came up, MF and JBK have noted that extra severity leves are **NOT** the right enhancement to meet this need. Instead, we have suggested a form of debugging messages; see Enhancement 013. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ Superceded by 013. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- ErrorLogger 011: Documentation index Statement of suggested enhancement 7/31/02 ---------------------------------- version 2.1.9 It was noticed by mf and jbk that the documentation for ErrorLogger, while complete, is hard to navigate. An index would be nice. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- mf is going ahead with this. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- ErrorLogger 010: Map of DestControl handles Statement of suggested enhancement 6/7/01 ---------------------------------- version 2.1.4 In a ZOOM Postmortem on the ErrorLogger package design, it was brought up that if the handle to the ELdestControl object returned by attach() is lost, there is no way to get it back. Everyone that uses ErrorLogger extensively has this happen. The solution that would make users happy would be to give a string name to the thing, and an interface that wold allow it to be searched for. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- One tough interface issue is what the string name would be. One easy way out is to provide another signature for attach, taking a string. One could try to assign a string instead, by some set of rules (ELoutput to a file would use the file name, ELstatistics would use "statistics", ELoutput to a stream would use ... what??? If we did this, getting the name right would still be guesswork on the users' part. So let's just keep it simple. The other tough issue is what to do if the user seraches for a string which is not found. This is discussed below. Some details -- Consistant with the way other in-principe-unlimitted maps are used, one might want to have a data type ELhandleMap defined as a map of strings to handles, so that if a frameworker wants to limit map growth he can substitute another limited class. But I claim this is not needed: First, attaching destinations is itself done at a setup time, when memory growth is OK, and second, the number of destinations will generally be quite small; 10 would be unusually large. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- We will implement this ADDING one signature for ELadministrator::attach() and a method of ErrorLog (and ELtsErrorLog) which forwards to ELadminsitrator:: getDestControl. The syntax will look like: // in the setup code ELdestControl logfile; logfile = logger->attach(ELoutput ( "filename.log", "myLogFile" ) ); // in later code, where that logfile has been lost ELdestControl logfile; bool handleFound; handleFound = errlog.getELdestControl ("myLogFile", logfile); if (!handleFound) { ... oops, wrong name -- user chooses what to do here } (A second signature, without handleFound, could be supplied for the lazy. I'll decide, for now, not to suppliy that.) In the wrong-name case, a severe error will be declared on errlog, but in case the code will continue, the returned logfile would be the default ELdestControl, which has d = 0. I will modify ELdestControl.cc so as to make all control operations nops when d = 0. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ Unless some user comes up with a reason not to do this - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done 4/23/02 ---------------- In version 2.1.8 ================================================================== ErrorLogger 009: nobr Statement of suggested enhancement 6/7/01 ---------------------------------- version 2.0.8 M. Fischler noticed that the nobr object, which is documented in Further Options for the Frameworker / What the error message output will look like as "not yet implemented" is indeed not implemented. We should consider implementing this (or if not, remove the feature from the documentation. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- Probably the only thorny issue is what type of object "nobr" is. Given the fact that our users are not relying on namespace (yet?), nobr is a bit short to want to pre-empt the symbol. 8/3/01 The nobr feature would need to be implemented at the ErrorLog.icc level, in much the same way as endmsg was. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- Unless we get some specific request, I am inclined simply to eliminate this feature. And if we do implement it, I am inclined to call it something more EL specific, like ELnobr, - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ 8/3/01 The nobr feature would need to be implemented at the ErrorLog.icc level, in much the same way as endmsg was. Thus we are involved with defining a new class type, entwined in a template mechanism, and so forth. It would also require having a new one-shot data member to remember the nobr status, and addressing the issue of what happens if a message leaves a dangling nobr. Since no users have been requesting this feature (even though the documentation practically begs one to ask when it will be available, I intend to simply obviate the feature. This proposal will be put on the enhancements page, and barring objections, will be done on the next release. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- 11/29/01 (Documentation was fixed to reflect nobr elimination) ================================================================== ErrorLogger 008: Thread-safety Statement of suggested enhancement 5/22/01 ---------------------------------- version 2.0.6 Gordon Watts needs a way to use the ErrorLogger package in a multi-threaded application. The current implementation garbles messages if two threads are preparing messages at once. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- thread A thread B mylogger (ELwarning, "Yuck"); | otherlogger (ELinfo, "Yes"); mylogger << "Real Trouble"; | otherlogger << "Some info"; mylogger << endmsg; | otherlogger << endmsg; is going to be garbled. Here are the key points: a) There is a singleton ELadministrator: for our purposes, a singleton is an object that has a **static** pointer and a static method instance(); thus all threads will be sharing the same ELadministrator. b) The ELadministrator has an ErrorObj member, which is a box where the parts of the current error message are collected as they come in. c) The ErrorLog has a pointer to the ELadministrator. d) When you do errlog (severity, "id") and when you errlog << item, this adds data to the ErrorObj owned by the ELadministrator which errlog knows about. This is a point at which the existing code is fouling you up: Thread 1 starts a message, thread 2 starts a message (thus forcing completion of the message from 1), thread 1 adds to its message (which addition goes to the message from thread 2) and so forth. e) The ELadministrator has a list of ELdestination sinks to which to send the completed message. f) The ELcollected destination is thread-safe, if the user-supplied ELsender method uses locks to avoid clashes between two sent messages. g) The ELoutput definition cannot be made thread-safe since it is sending data to (assumedly) the same stream. A stream, or at least ostreams like cout and an ofstream associated with a file, is effectively a global concept. ELoutput thread-safety must be acoomplished in one of two manners: i) Only one thread is attempting to write to that ELoutput; or ii) The use of the ELoutput is protected at a higher level (say at the ELadministrator level) by a lock. One possible solution is Thread starts a message to errlog **either** by presenting a fully formed ErrorObj **or** by errlog(sev,"id") This obtains the "I am writing an error message" lock possibly, thread blocks till that lock is available Thread may add more to the message Thread eventually does endmsg if the << form is used. ELadministrator calls the emit() methods of all ELdestinations **before** the lock is release; thus the streams are never mixing parts of different messages ELadministrator deletes the "I am writing an error message" lock To do this, say we provided (in analogy with the ELsender idea) a base class class ELmutex { virtual void obtain() = 0; virtual void relinquish() = 0; } and the ELadministrator (or the ErrorLog class) owns an ELmutex* which starts as 0, and one new method setMutex (ELmutex *) Then the crucial code change in the ELadminstrator is two lines in two places: When a message is started: if (mutex) mutex->obtain() When the loop of ELdestinations is completed: if (mutex) mutex->relinquish() -- However, this has the drawback (pointed out by Jim Kowalkowski) of not being exception-safe. The fix for that (pointed out by Marc Paterno) would be to move things around such that the entire locked period can be in one block; then the "acquisition is construction" metaphore can be used. For ErrorLogger, that implies moving the code to accumulate items into the ErrorObj out of ELadministrator and into ErrorLog. This is not particularly tricky, but of course it has some potential of fouling up. And it forces the user to instantiate a separate ErrorLog for each thread; and there is no easy way within the thread to treat that as a global so one has to worry about how to pass around errlog. (Decent ways of doing this do exist, though.) Another way to achieve exception safety is to insist that (in the multi-thread application) all use of the ErrorLogger is via creating an ErrorObj and then passing it to the errlog. In that case, we can isolate the critical code to one routine. -- Another possible solution is via ELcollected destinations: If (a) each thread can instantiate its own ELadministrator (in a completely separate way such that they do not share their static data - that is, defeating the singleton mechanism); each errlog would be instantiated telling it to use the admin for its own thread (b) the various destinations which would be ELoutput are now made into ELcollected destinations (see "collective error logging" link off the ErrorLogger main page) -- the transport mechanism should be easy among threads (c) one further thread is created to receive the collected messages and producce the actual ELoutput log If this can be done, then everything is inherently safe. The ELcollected destination was meant for this sort of purpose (though one envisioned the threads as separate processes at the time). The sender class inheriting from ELsender is written by the user and contains all the mutex mechanism in one point. This burdens the user with attaching an ELcollected in each thread, and creating an ErrorLog in each thread -- and the usual headache of how to pass errlog around still comes up. One big advantage, though, is that if the code every migrates to multi-processor, the only code that needs to change is the sender mechanism. -- It is not yet clear which of the four solutions to choose: A) Lock options in the ELadministrator (least code modification but not exception safe). B) Move ErrorObj accumulation into the ErrorLog (thread-safe but a fair amount of modification to the package code). C) Restrict mutex-involved issuing of messages to use the errlog << ErrorObj form, and put exception safe locking into the code handling that case. D) Provide a way to have multiple ELadministrators, and attach errolgs to them; then the user uses the ELcollected mechanism. -- It has subsequently been pointed out by W. Brown, with concurrence of M. Paterno, that the best way to do this is by introducing a templated class which I would call ThreadSafeErrorLog < mutex >. This makes it cimple to provide exception safety and also to avoid changing the code path executed by existing applications. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- At this point, MF is going ahead with coding the new ThreadSafeErrorLog class. Barring unforseen snags, it will be strongly suggested that this be incorporated. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ mf -- * will create the new class (this is starting now) (1.2) * will test new class in 1-thread mode ( .5) * will test the original functionality and commit ( .2) Some testing will be done in conjunction with G. Watts. * will learn the use of pthreads in one environment (1.0) * will create a serious test case (1.5) * Documentation and release ( .3) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Status of implementation ------------------------- Completed (6/29/01) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done 7/11/01 ---------------- Appears in V2.1 (release R-01-07-11) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Location of Documentation ------------------------- From the main ErrorLogger documentation page, look at the link (bottom list) to "Error Logging for Multithreaded Processes". ================================================================== ErrorLogger 007: Multiple module inclusion/exclusion Statement of suggested enhancement 3/15/01 ---------------------------------- version 2.0.4 Liz Sexton-Kennedy writes that users want to be able to filter messages by a module list, not just one. In particular, they have interactive users who would benefit by being able to flexibly control filtering. A suggested interface would be addExcludeModule() addIncludeModule() clearExcludeModuleList() clearIncludeModuleList() and if both an include and an exclude list were present, the behavior would be undefined. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- Probably the toughest part is to decide on the semantics. My suggested interface would be as follows (these are all methods of ELdestControl): dest.ignoreModule("*"); Clear `respondToModule' list, and filter out all messages, except those coming from any module that is later added to the `respondToModule' list. dest.respondToModule("name"); Add this module to the `respondToModule' list, and remove it (if present) from the `ignoreModule' list. dest.ignoreModule("name"); Add this module to the `ignoreModule' list, and remove it (if present) from the `respondToModule' list. dest.respondToModule("*") Clear `ignoreModule' list, and respond to all messages, except those coming from any module that is later added to the `ignoreModule' list. Thus the current dest.filterModule("modname") would be equivalent to (and re-implemented as) dest.ignoreModule("*"); dest.respondToModule("modname"); while the current dest.excludeModule("annoyingmod") would be equivalent to (and re-implemented as) dest.respondToModule("*"); dest.ignoreModule("annoyingmod"); And the behavior if a user does both dest.ignoreModule("*"); and dest.respondToModule("*"); or specifies some ignores and some RespondTo's is well defined: The last of the "*"'s toggles which list to pay attention to (as well as clearing that list), and adding a module to the list which is being ignored is effectively a nop. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- Liz Sexton agrees on the proposed interface, and at meeting of 4/4/01 it was decided to go ahead/ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ mf-- * make the changes (.3) * Will design & apply test cases (.3) * Documentation and release notice (.1) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Status of implementation ------------------------- Enhancement is completed and tested (4/5/01) Estimated check-in date 4/9/01 Estimated release date 4/15/01 (probably together with Exceptions enhancements) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done 4/5/01 ---------------- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Location of Documentation ------------------------- From the Framewoker basics page of the ErrorLogger documentation, link to Filtering by Module or Multiple Modules. ================================================================== ErrorLogger 006: DiscardThreshold for ErrorLog Statement of suggested enhancement 3/7/01 ---------------------------------- version 2.0.3 Matthew Worcester writes that he has code with a lot of error logger statements set below the ELerror threshold (bank dumps and such that are useful for expert debugging) that (would) put a lot of information on the screen. It's easy for the experts running this code to set ELerror to ELinfo and see all the relevant info. However, these seem to be slowing the code down considerably, even when the theshold is set so that the messages ... "Is there some way to prevent this? When I remove these ELinfo statements, my code runs 4-6 times faster, but now I don't have easy access to my "expert" level information!" - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- I traced the execution logic, and indeed a fair amount of work is done even if the message is ultimately ignored. The estimate I come up with is about 7 usec, plus 1.5 usec per destination that has been attached, plus 1.7 usec per item << to the log. So for example, errlog (ELwarning, "this is") << "some text" << 45 << " more" << endmsg; will eat perhaps 15 usec. Although I clearly never felt this to be of concern, equally clearly it is to this user. Fortunately, a way to cut this be a factor of about 30 is very easy to do. I propose to add the following method to the ErrorLog class: void setDiscardThreshold (const ELseverityLevel &); The workings of the method are as follows: If the discardThreshold for a logger theLog has been set to some level sev, the processing of any message assigned a severity of less than that threshold will be terminated AS SOON AS POSSIBLE. The message is discarded and will not make it to statistics destinations, to ELcollected, or to anywhere else. Further, the logger will have a flag set which tells it to immediately discard any items sent it via <<. This is in effect till the next operator() gives a message with sufficient severity. The threshold applies only the that specific ErrorLog (theLog); if the user has several logs active they may each have a discardThreshold. One may worry about the case where there is an unterminated (no errmsg) message active and one of these ignored messages comes in. We could afford to check for message_is_active and finish the other message in that case. However, it is best not to do that: This way, the behavior with respect to all non-discarded messages is unaffected by the insertion of a discarded message. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- At ZOOM meeting of March 7 it was decided to go ahead another enhancement. As long as their will be a release, this might as well be included. I wll post this on the (new) enhancement discussion page but if nobody points out any objections, I will go ahead with it. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ mf-- * Will make the changes (.2) * Will design & apply test cases (.1) * Documentation and release notice (.1) * To be done in concert with 003 and 004 Estimated cvs checkin date 3/14 Estimated date of release V2.0.1 3/16 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Status of implementation ------------------------- Done and checked in (3/13/01) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done 3/15/00 ---------------- version 2.04 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Location of Documentation ------------------------- In ErrorLogger-->Frameworkers Basics-->Rapid Discarding Below a Severity ================================================================== ErrorLogger 005: Hex output option for ELoutput Statement of suggested enhancement 3/07/01 ---------------------------------- version 2.0.1 Matthew Worcester writes that the experts complain that they would like some numbers in hexadecimal, which is easy in cout, but doesn't seem to work in the error logger. Is there a way to get hex info in the error logger? This is about the 4-th person who requested this. In the past, we just said "use an ostringstream to format your number as a string," and this was OK when the people affected were few, but perhaps we should try to supply an accomodation. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- The use of std:: manipulators (such as hex) is hosed because there is no standard for their signatures. We tried hard to be able to pass manipulators correctly and failed to be able to do this in a portable way. However, this lack seems to be affecting too many people for us to ignore it. I propose to add the following method to the ELdestControl class: void setHexTrigger (int m); Only ELoutput will react to this. The behavior is as follows: When an int (and only an int) is << to the ErrorLog, if the absolute value of the int is at least m, then the int will he printed out as both a decimal and an eight-digit hex number. For example, 65534 [0000FFFE]. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- At ZOOM meeting of March 7 it was decided to go ahead another enhancement. As long as their will be a release, this might as well be included. I wll post this on the (new) enhancement discussion page but if nobody points out any objections, I will go ahead with it. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ mf-- * Will make the changes (.2) * Will design & apply test cases (.1) * Documentation and release notice (.1) * To be done in concert with 003 and 005 Estimated cvs checkin date 3/13 Estimated date of release V2.0.1 3/16 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Status of implementation ------------------------- Done and checked in (3/14/01) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done 3/15/00 ---------------- version 2.04 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Location of Documentation ------------------------- In ErrorLogger-->Further Options for Frameworker--> Hex output formatting via ErrorLog ================================================================== ErrorLogger 003: statisticsMap method for ELstatistics Statement of suggested enhancement 2/22/01 ---------------------------------- version 2.0.1 Wolfgang Wagner writes that he was thinking about how to collect the statistics of errors of one particular run for each consumer and write this into a data base. To do this I wanted to use the ELstatistics class... From the statistics summary I can get a string with all information in it. However, to get the single numbers I would have to parse this string. It's not to difficult but a bit tedious. I'm writing to you because I was wondering whether it would be possible to implement an access method to ELstatistics which returns a copy of the ELstatistics map. If this was possible I could simply access the individual error messages and retrieve the statistics information. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- The ELstatistics map does not contain whole messages, just ID's. But that is probably what W.W. meant, since that is what comes out in the summary anyway. I don't want to return "a copy of the ELstatistics map," since that says that std::map will forever be used as that internal data structure: And we took considerable pains to make it possible to substitute something else (without open-ended memory use) for the map -- so that during a run all the memory allocation can happen at the start or at termination. But I have no problems with supplying something which is explicitly stated to be an std::map . These are already public data types (and map is already pulled in by the ELstatistics header via ELmap.h. I propose to add the following method to the ELstatistics class: virtual std::map statisticsMap() const; This returns a **copy** of the statistics data, in the form of a map, even if in the future some version of ELstatistics keeps the data internally in some other form. I will send this out for comments... - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- At ZOOM meeting of March 7 it was decided to go ahead with this. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ mf-- * Will make the changes (.1) * Will design & apply test cases (.1) * Documentation and release notice (.1) * To be done in concert with 004 and 005 Estimated cvs checkin date 3/13 Estimated date of release V2.0.1 3/16 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Status of implementation ------------------------- Done and checked in (3/14/01) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done 3/15/00 ---------------- version 2.04 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Location of Documentation ------------------------- In ErrorLogger-->Further Options for Frameworker--> Getting ELstatistics information as a map ================================================================== ErrorLogger 001: Output Header Suppression Option ("No-established") Statement of suggested enhancement 2/6/01 ---------------------------------- Current version 2.0 A way to inhibit the "Error Log established" message, so that error-free code can send absolutely nothing to an output desitnation. Robert Kennedy (CDF) writes: I am adapting various small CDF EDM command-line utilities to use the ErrorLogger. These utilities now produce no cout/cerr output if there are no errors encountered under normal operations, which I would very much like to reproduce as users are accustomed to this intuitive behavior. The problem I have is that there is no straight-forward way to bypass the message emit()'d by the ELoutput constructor: ======================================================= Error Log established 6-Feb-2001 11:49:23 ======================================================= ... If I pass a dummy output stream to the constructor to "sink" this message, then I cannot later assign a real output stream to ELoutput And deriving a class from ELoutput simply to avoid this output is silly. Is there some alternate way to avoid all output *unless* an EL message above threshold is logged and still have the error logging under user/program control? An additional bool argument to the ELoutput constructor to permit the suppression of this message would suffice, with the default behavior left as is. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- Rob is right, this would have to be done in new signatures of constructor(s). There are three relevant constructors to (default, ostream&, and filename). Probably the most straightforward method would be to add a bool argument, defaulting to false, to each constructor, and place simple logic into those three places to skip the message. Some background: This message was found to be very useful while testing the package. If an already-existing file is specified, it is opened for append, so as not to wipe out possibly valuable information from previous runs. But we would often forget to remove a log before a test -- the "established" message let us find where the new output started. This consideration is not important in cases where the normal expected log would be empty anyway. RK did not request the following capability which would make the implementation somewhat less trivial: If the "established" messge was supressed, and then a message makes it out to the log, should the "established" message no be output preceding it? And if so, do we need to remember the date and time string of actual establishment. I (mf) prefer not to open these issues -- if you supress that establishment message, that is that. Discussion: It was agreed at a ZOOM meeting that the change ought to be done, by adding defaulted arguments to some constructors, but that we should not fool with the default constructor: That is more risky, plus the requestors are not using the default form of the constructor anyway. It was agreed that defered output of the "established" message iff any normal message gets emitted by this output would be of some marginal value but was not worth the complexity cost. Risk assessment: The right way to implement this involves adding a defauted argument to each relevant constructor; if this has a non-default value the message is to be supressed. Any time you change a constructor, the risk is greater than that of adding a new method. Here, however, the change is very minimal. Testing is also relatively straightforward. This, then, is a low-risk change. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- This change will be made: in class ELoutput -- ELoutput ( ostream &os ) becomes ELoutput ( ostream &os , bool supressEstablished = false ) ELoutput ( ELstring filename ) becomes ELoutput ( ELstring filename , bool supressEstablished = false ) and the appropriate implementation change will be made in ELoutput.cc. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ mf-- * Will make the changes (.2) * Will design & apply test cases (.2) * Documentation and release notice (.2) * Lower priority than D0/NT efforts Estimated cvs checkin date 2/13 Estimated date of release V2.0.1 2/15 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done 2/14/01 ---------------- Checked in 12/14/01. Will be version 2.0.1, released 2/15/01. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Location of Documentation ------------------------- Go to the ErrorLogger main documentation page. Click Further Options for the Frameworker on the list near the bottom. Click ormatting control available in ELoutput. Observe the last feature documented in that section. ================================================================== ================================================================== ================================================================== Suggested Enhancements Not Made ------------------------------- ErrorLogger x004 De-scoped DestControl Statement of suggested enhancement 2/27/01 ---------------------------------- version 2.0.1 Gustaaf Brooijmans has a situation in which he allows ELdestControls to go out of scope (between two runs) but later wishes to control those destination, e.g., to call zero() on these destinations at the start of a run. "So, I'd like to be able to access the corresponding ELdestControl. It seems to me that the best way to do this would be if I could get at them through a method of the ELadministrator, but that doesn't exist." "Another reason I would like that is that I would like to be able to destroy a destination..." The intent here would be to wipe that destination off the list known to the ELadministrator; just allowing the ELdestination and the ELdestcontrol will not do this, as the administrator owns a clone. If he could eliminate a destination, he would feel comfortable opening and attaching a new destination of that same type for each run. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- There is logically no way to have a method of ELadministrator to recover a handle to a destination on it list of sinks, becaue how would you specify **which** destination you meant. You need some sort of handle, and as long as you are saving such a handle, it might just as well be the ELdestControl. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision NO GO -------- In a phone discussion of 2/27/01, GB agreed that the issue of needing a handle makes the idea of a recover-the-ELdestControl method handle unfeasible. MF suggested making a singleton class to hold the ELdestControl past the lifetime of the run object; GB noted that he already had such a singleton and could just add the ELdestControl to that. As to eliminating a destination: The intended means of disabling a destination is to set its threshold to ELhighestSeverity. It is conceivable that the small overhead of checking threshold could in some application become an unacceptable performance burden (but note that this happens only on issuance of a ErrorLog message, and the check is very fast). If that situation comes up we can reconsider making a method ELdestControl::zap(). It would be very wrong to tie the elimination of a destination to letting its ELdestControl go out of scope; that invalidates the intended common basic way of using the ErrorLogger package! But if we ever do provide a zap() method, we will have to deal with the fact that you are then left with a "dangling" ELdestControl. Given that the ErrorLogger design does allow for disabling a destination, and that the reasons for not going futher than that were (and remain) valid, we will stick to the design in this matter. ================================================================== ErrorLogger x002 A const log method Statement of suggested enhancement 1/31/02 ---------------------------------- version 2.0 Gustaav Brooijmans points out that the methods of and ErrorLog to emit a message are non-const. So if a class uses its own private ErrorLog object (which is a recommended mode of use) no const methods of that class can send error messages. A good solution exists in C++, but > > I'm not hot on declaring the ErrorLog object mutable. Is there a better > way out of this? > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- Since the usual semantics of operator<< are inherently non-const, it would be wrong and potentially troublesome to try to make this const. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision NO GO -------- Since there is a proper solution in C++ using mutable, and since all the compilers we are even hinting at ever using implement mutable, we should not change the package to work around this. None of the compilers that we use around here or are even considering using have the DEFECT_NO_MUTABLE. (Evidently that was an easy feature for all the vendors to implement.) So declaring _my_errlog mutable is not a risky solution. ================================================================== Template for each suggested enhancement:: ================================================================== EnhancementNumber: Enhancement Name Statement of suggested enhancement ---------------------------------- version x.y * * * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Technical Information --------------------- circumstances rsik assessment * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Decision -------- * * (or technical problems) * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Plans and Priority ------------------ * * * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Status of implementation ------------------------- * * (in long-running problems might need more than one) * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Enhancement Done ---------------- version x.z - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Location of Documentation ------------------------- ==================================================================