A Legacy Notes Developer's journey into madness.

A little bit of scope will never. Never, ever, ever erase.

Devin Olson  August 10 2012 10:54:04 AM
Last week I ran into this little nugget of fun:  
Unexpected runtime error
The runtime has encountered an unexpected error.
Error source

Page Name:
/request.xsp

Exception

java.io.NotSerializableException: lotus.domino.local.Document
lotus.domino.local.Document




The problem was entirely mine, and caused by my attempt to store a non-serializable object in viewScope.


What does that mean?  If you will indulge me for a bit, I will try to explain.  Please remember that I'm a beginner, and may make some statements that turn out to be incorrect.  If I later discover something is wrong I will come back and correct it.

Let's start with scope.  Excerpted from Dictionary.com*:
scope [skohp] noun, verb, scoped, scop·ing.
noun

1.        extent or range of view, outlook, application, operation, effectiveness, etc.: an investigation of wide scope.
2.        space for movement or activity; opportunity for operation: to give one's fancy full scope.
3.        extent in space; a tract or area.
4.        length: a scope of cable.
5.        aim or purpose.

origin:

Neo-Latin
-scopium  < Greek -skopion, -skopeion,  equivalent to skop ( eîn ) to look at (akin to sképtesthai  to look, view carefully; compare skeptic) + -ion, -eion  noun suffix.  

"extent," 1534, "room to act," from It. scopo "aim, purpose, object, thing aimed at, mark, target," from L. scopus, from Gk. skopos "aim, target, watcher," from PIE *spek- "to observe" (cf. Skt. spasati "sees;" Avestan spasyeiti "spies;" Gk. skopein "behold, look, consider," skeptesthai "to look at;"

L. specere "to look at;" O.H.G. spehhon "to spy," Ger. spähen "to spy"). Sense of "distance the mind can reach, extent of view" first recorded c.1600.


programming

The
scope of an identifier is the region of a program source within which it represents a certain thing. This usually extends from the place where it is declared to the end of the smallest enclosing block (begin/end or procedure/function body). An inner block may contain a redeclaration of the same identifier in which case the scope of the outer declaration does not include (is "shadowed" or " occluded" by) the scope of the inner.


In computing the scope defines the boundaries within which something is available.  If a portion of your running program or code can directly access something (I'm being deliberately generic here, as "something" can be "anything" - an object, a variable, a subroutine, a function, a data source, etc), then that something is said to be "in scope".  If that something cannot be directly accessed, it is "out of scope".  There are various ways to get information from or send information to things that are out of scope.    Consider the following example.

Create in you mind two computer methods.  Use whatever language you are best able to think in -this is an example and we don't need to concern ourselves with semantics.   Both methods will accept a numeric variable, perform some operation upon it, and return the result of that operation.  

Internally, the first method will determine the number of seconds that have elapsed since midnight and assign that value to an internal variable called "secondsSinceMidnight".   It will also store this value the FIRST time it is called in a variable called "secondsWhenStarted".  These variables are not referenced anywhere else in your program other than within the first method.   The first method will write these values to some form of log, then add the value of "secondsSinceMidnight" to the passed in parameter and return the result.  

Internally, the second method will create a variable called "resultValue", and will assign a value to it by calling the first method and passing the parameter to it.   It will then write that value to the log and then return.

These two methods and the program that calls them could look something like this (made up language):

method
myFirstMethod(number arg0) returns number [[
static number secondsWhenStarted
number secondsSinceMidnight
secondsSinceMidnight = day.elapsedSeconds()

if (not initialized(secondsWhenStarted)) [
 secondsWhenStarted =  day.elapsedSeconds()
]

printToLog(secondsSinceMidnight, secondsWhenStarted)
return(secondsSinceMidnight + arg0)
]]

method
mySecondMethod(number arg0) returns number [[
number resuiltValue
resultValue = myFirstMethod(arg0)
printToLog(resultValue)
return(resultValue)
]]

program
myProgram(array arguments) [[
number myNumber
number = 1
repeat[
 printToLog(mySecondMethod(number))
 number = (number + 1)
] until (number > 4)
]]

If this program starts at 1 second after midnight, and assuming that it takes 1 second for each iteration to complete, the results of the log might look something like this:

00:00:01 myFirstMethod: 2, 1
00:00:01 mySecondMethod: 2
00:00:01 myProgram: 2
00:00:02 myFirstMethod: 3, 1
00:00:02 mySecondMethod: 3
00:00:02 myProgram: 3
00:00:03 myFirstMethod: 6, 1
00:00:03 mySecondMethod: 6
00:00:03 myProgram: 6
00:00:04 myFirstMethod: 8, 1
00:00:04 mySecondMethod: 8
00:00:04 myProgram: 8


So what does this have to do with scope?   Everything.  For code running inside the fist method, the variables "secondsSinceMidnight" and "secondsWhenStarted" are directly accessible, the code can create, read, update, and destroy them.  They are in scope.  As soon as the running code steps outside of the function, such as within the second function or the main program, these variables go out of scope.  Neither the main program or the second function can directly access these variables.  

I keep using the term "directly access" for a reason.  Consider the variable "resultValue" in the second method.  This value is set to the result of the first method, and that result is determined by a combination of code within the first method and the parameter passed to the first method.  It is directly accessible only from within the second method -neither the first method or the main program can see it.  It is in scope only within the second method, and out of scope elsewhere.  Code in the main program can cause a change to its value (by passing in different parameters), and code in the second method can cause a change in value (by returning different results), but neither of these changes happen directly.  They are dependent upon code within the second method to complete this change.  The important point here is that code can effect out-of-scope variables -but only indirectly.  
Ok, so we have touched on scope a bit.  There is a whole world of exploratory discussion available bout it, but it is beyond the scope of this post ;-)  


Back to the viewScope issue.  

The XPages environment has 4 very special repositories for developers to use.  These are applicationScope, sessionScope, viewScope, and requestScope.  Even though all of them end in "Scope" they are, from a pure computer science point of view, not really scoped variables.  They follow scope rules to a point, but because of the limitations of the underlying JSF architecture there are some gotchas that need to be understood.  These variables are Maps, in that they descend from a java Map object (technically they are a com.sun.faces.context.xxxxx objects).  

You may have heard something analogous to "you can put anything into a Map".  While not technically true this is a pretty close description.  I find it easier to think of a map less as a container and more of a collection of pointers to other things.  With XPages these 4 maps help compartmentalize things for in a manner that closely approximates true scoped variables, which is why they are called what they are called.  They do not contain every variable within the scope of your particular program code -only those things that are put into them.   They are globally accessible Maps that allow us to create, read, update and delete things within the scope as defined by the prefix of the map name.   The name of the Map helps us understand how long the things we map will exist before being automatically destroyed by the garbage collector.

requestScope:

Things last for the duration of the GET or PUT processing only.  Once that processing has finished the elements will be garbage collected.

viewScope:

Things last for the duration of the XPage being viewed  (look at the source of any XPage control you have.  Everything between the "xp:view" tags is within the viewScope.  These define the boundaries.  


sessionScope
:
Things last for the duration of the User Session.  From the time the user logs into the application (or possibly connects as Anonymous -this is an area I'm not entirely certain of) until the user leaves.  Things in sessionScope are available as the user crosses pages.


applicationScope:

Things last for the duration of the Application in memory on the SERVER (not the browser client).  Things in applicationScope are available to multiple users (sessions), pages (views) and requests.  I know that things in applicationScope will eventually die and become garbage collected, but I'm not yet fully up to speed on what/when triggers this to occur.  


The thing to remember about these objects is that they can only store (map) serializable elements.    A serializable element is something that can be transmitted serially to another system and be fully usable by that other system.  The common test for such an element is: Can it be written to a disk and then retrieved with the FULL CONTEXT of the element / object available?  A string is serializable.  An integer is serializable.  A Notes Object (such as a NotesView or NotesDocument) is not.  You can convert information contained by or about the object to a serializable format, but you cannot serialize (at least without a LOT of effort) the object itself -because some context information (methods, references to other objects, etc) will be lost in the process.  This is the critical thing I forgot when I coded my method.


Back to my error.  Here is the code that caused the error:
function
getDocumentByUNIDfromDb(source:NotesDatabase, universalID:String):NotesDocument {
/*
getDocumentByUNIDfromDb
Gets a document from a database.
Caches the result in viewScope

@param source: Database from which to get the document.
@param universalID: UniversalID of the document to retrieve.
@return: NotesDocument for the universalID
*/
if
(@IsBlank(universalID) || (source == null)) { return null; }

var
tag:String = "document.repunid." + source.getReplicaID() + getDefaultDelimiter() + universalID;
if
(viewScope.containsKey(tag)) { return viewScope.get(tag); }

var
result:NotesDocument = source.getDocumentByUNID(universalID);
viewScope
.put(tag, result);
return
result;
} // getDocumentByUNIDfromDb  

My problem was caused by trying to grab the document from the viewScope object.  Interestingly, it worked fine when called the .put() method, and didn't blow up until I called the .get() method.   My attempts at performance optimization combined with my not thinking about the serialization issues are the core of this problem.  I will consider this an important lesson well learned.

There is one weird thing about this.  Currently you CAN store Notes Objects in requestScope.  If you take the above function and replace all instances of "viewScope" with "requestScope" it will work just fine.  I don't know why, although I suspect that it has to do with the duration of the request event and that for the requestScope map the underlying C object (which is what is ultimately what is being reference here) does not get garbage collected until the termination of the request event.  But just because you can do something doesn't mean you should.  In this case I think it is best to just live by the rule of: Never "scope" Notes Objects

-Devin.


* Dictionary.com, "scope," in Dictionary.com Unabridged. Source location: Random House, Inc. http://dictionary.reference.com/browse/scope. Available: http://dictionary.reference.com.