Add an isDirty attribute to your entity

Here is a useful Groovy expression that can be used to add a transient attribute to your Entity that will return true if any attribute in your entity is new or has been modified

 (entityState != 1)

See Image below

Then add this attribute to your view object and use it in an expression to highlight rows that have been modified and not yet commited,.

 inlineStyle="#{row.isDirty ? 'background-color:#FFF9B3;' : ''}"







 Here's another Groovy expression that will display the original posted value of any attribute in your entity that has been modified but not yet posted

adf.object.getPostedAttribute(adf.object.getAttributeIndexOf("Name"))

For an overview of using Groovy expression in ADF see this white paper by Grant Ronald

How to cancel a long running query (Part 2)

Earlier this year I wrote about a possible way to cancel a long running query from the UI

       see How to cancel a long running query from the UI

 I finally got around to  experimenting with this more and have created a sample app that can be downloaded from here 

A screenshot is below.   The sample was created with jdev 11.1.1.4 but It should work with 11.1.1.6 without any problems. This demonstrates the creation of a dynamic view displayed in a dynamic table and includes the cancel button.    You can type any valid sql into the input box and the table will display all of the columns returned.   If you create a particularly slow query you can test the cancel query button and you should get the error message displayed below.  Obviously you wouldn't want to give the user the ability to provide the SQL for the dynamic query but it makes for a good example of how a dynamic query can be created and displayed.

Since ADF blocks all requests going through the faces servlet while the server is processing the query,  I implemented the cancel button using a client listener that cancels the event then creates it's own ajax call to a plain old servlet.    That servlet retrieves the view object (if it exists) from a transient map object stored on the session by the CustomViewObjectImpl class and calls cancelQuery() on it.  If the cancel is successful the executeQuery method will throw an exception that is displayed in the UI.    

When you set a query timeout on the view object, ADF will spawn a monitor thread that sleeps for the timeout period.  When the thread wakes up it calls cancelQuery() on the ViewObject.    This is the very same thing the servlet is doing except it is initiated by the user instead of the monitor thread.
  
The view object is stored in a transient map so that Serialization will not be attempted however the view object is only stored on the Session for the duration of the execute method and then removed so it should never attempt serialization anyway.


Note this will not work in a clustered environment not configured for sticky load balancing.   Obviously the request must make it to the same server where the query is running in order to cancel it.  I don't know of an easy way around that.  Ideally it would be nice if you could send a command to the database containing a key to the session needing it's query canceled then it wouldn't matter which server the cancel request arrived on. That feature is not available apart from the Linux command line.


This sample app will cancel all queries running on the users session if for example the user had multiple windows/tabs open, each running their own queries (not too likely) but you can pass up a key on the cancel request if you want to limit the cancel to a specific query.  

Note:  I initially could not get the cancel to work when running on Windows with the integrated development environment.  It ran fine however when I deployed it to the server running Linux.  
The query timeout setting was also not being honored when running under Windows.   I know this used to work for me so I'm thinking there must have been a network configuration change or upgrade that broke it.  I then discovered that I could get the cancel to run under Windows by adding the following JVM startup parameter to the "Launch Settings"

 -Doracle.net.disableOob=true 

This disables "Out of Band" breaks and instead uses "In Band" Breaks.  The sample app is configured this way but you may or may not need it depending on your network drivers and configuration.

Feel free to comment if you find this useful.

To progress this concept further I would like to add a page to our monitoring application that will iterate over the sessions and display a list of all long running queries and the users that are running them.

 

 

 

Allowing Read-Only Access to your application

We recently had a requirement to allow user logins with any assigned role but with read only access to the application.   User should be able to login and see all of the pages associated with the assigned role but cannot do anything that will add or change data in the database.    This turned out to be fairly simple to implement.   We created a new role "readOnlyAccessAllowed".  We already have a custom base Application Module class that all of our Application Modules extend.    So the change was simply to override the beforeCommit method, check if the user has the read only access role and throw an exception if so.  We also display a tag on the home page to assure the user that they have read only access.

If you do not already have a base class defined for your application modules, this is done by editing the Model.jpr  project properties.  Under Business Components/Base Classes you define a base class for your View Objects and Application Modules.   If you do this after your application modules have been defined and your app module has a java implementation, you will need to edit those java implementations and verify that they extend your new base app module.  Newly created app modules will automatically extend your base app module.

Below is sample code showing the implementation. The were 2 or 3 app modules used for storing user preferences, saved searches and login history that we still wanted to be allowed so I excluded those.  I also added a method allowOneTimeCommitForReadOnlyUser that can be called in the code immediately before calling commit() to allow for other exceptions to the rule.

    /* overriden in order to prevent commiting data when user has read only user role
     * beforeCommit is called for both the root app module and the app module
     * containing the view object being commited.
     */
    @Override
    public void beforeCommit(TransactionEvent e) { 
        boolean isReadOnlyAccess = getDBTransaction().getSession().isUserInRole("readOnlyAccessAllowed");
        String appModuleName = this.getRootApplicationModule().getName();
        // allow commits for login history, performance monitorng and user preferences
        boolean allowCommit=(!isReadOnlyAccess || 
                             "AppCtrlAM".equals(appModuleName) || 
                             "SearchAM".equals(appModuleName)   ); // allow readOnlyAccess users to save searches criteria
        // this is used to allow writing the account audit info for USER_ACTIVITY table       
        if (allowOneTimeCommitForReadOnlyUser){
          allowCommit=true;
          allowOneTimeCommitForReadOnlyUser=false;
        } 
            
        if(!allowCommit){ 
            displayReadOnlyMessage();
            throw new ReadOnlyAttrException(0,"","",this.getName(),"");
        }
       
        super.beforeCommit(e);
    }
    
    public static void displayReadOnlyMessage(){
      FacesContext context = FacesContext.getCurrentInstance();
      FacesMessage fm = new FacesMessage(FacesMessage.SEVERITY_ERROR,"Read Only","You are allowed read only access.  Data cannot be added or changed.");
      context.addMessage(null, fm);
    }

    public void allowOneTimeCommitForReadOnlyUser() {
        ApplicationModule am = this.getRootApplicationModule();
        if (am instanceof CAAppModuleImpl){
          ((CAAppModuleImpl)am).allowOneTimeCommitForReadOnlyUser = true;
        }
        this.allowOneTimeCommitForReadOnlyUser = true;
    }

Here's a link to another solution.  That solution makes input fields display as read-only output fields.
We didn't do that because we didn't want to change the Application look in any way because we use Selenium non-invasive tests to regression test our pages in production so we want the page to look the same as it does for a normal user but still prevent the test or user from committing data.

Selenium WebDriver utility for determining when page has finished rendering

We are using Selenium for automated testing of our web app.   We use the new Selenium WebDriver API to create java based unit tests for interfacing with the UI.  One problem we had was determining when the page had completely finished rendering as the page may still be in the process up updating after the initial page is loaded.   There is an ADF javascript function for checking this

"AdfPage.PAGE.isSynchronizedWithServer()"  so

the helper method below can be called by the test class to determine when the page is fully synchronized with the server and the next action can be performed.



public static void waitForPageToFinishRendering(WebDriver oDriver, int timeout) {
        ExpectedCondition e = new ExpectedCondition() {
            public Boolean apply(WebDriver d) {
              JavascriptExecutor js = (JavascriptExecutor) d;
              Boolean isReady = (Boolean)js.executeScript("return AdfPage.PAGE.isSynchronizedWithServer()");
              return isReady;
            }
          };
          WebDriverWait w = new WebDriverWait(oDriver, timeout,100);
          w.until(e);
    }

For a Good overview of using Selenium with ADF see 

http://www.oracle.com/technetwork/articles/adf/part6-094560.html
That article talks about using waitForElementPresent before performing the next action but I have found using the technique above to be more reliable.  Especially for complex pages where there is a lot of lazy loading going on.

The danger of using the default settings of a view object for committing data

We recently had a problem in production where the weblogic servers randomly ( once or twice per day) became extremely slow (basically unusable) for a period of time before recovering.  We discovered that the server was  running low in memory causing a perpetual garbage collection cycle to occur.   This begin occurring after loading a large amount of data (several million rows)  that was transferred from our old application.  Up until this point the system had been humming along fine.  An analysis of the thread dump showed that during this time of crisis the server was looping in the following code.

                oracle.jbo.server.QueryCollection.hasNextInResultSet(QueryCollection.java:4611)

                oracle.jbo.server.ViewObjectImpl.hasNextForCollection(ViewObjectImpl.java:6899)F

                oracle.jbo.server.QueryCollection.hasNext(QueryCollection.java:4579)

                oracle.jbo.server.QueryCollection.populateRow(QueryCollection.java:3553)

                oracle.jbo.server.QueryCollection.fetch(QueryCollection.java:3387)

                oracle.jbo.server.QueryCollection.get(QueryCollection.java:2188)

                oracle.jbo.server.ViewRowSetImpl.getRow(ViewRowSetImpl.java:5016)

                oracle.jbo.server.ViewRowSetImpl.getRow(ViewRowSetImpl.java:3242)  

                oracle.jbo.server.ViewObjectImpl.activateTransients(ViewObjectImpl.java:18328)

                oracle.jbo.server.ViewObjectImpl.activateTransients(ViewObjectImpl.java:18289)

                oracle.jbo.server.ViewObjectImpl.activateState(ViewObjectImpl.java:18512)

                oracle.jbo.server.ViewObjectImpl.activateState(ViewObjectImpl.java:18407)

                oracle.jbo.server.ViewRowSetIteratorImpl.activateIteratorState(ViewRowSetIteratorImpl.java:4025)

                oracle.jbo.server.ViewRowSetImpl.activateIteratorState(ViewRowSetImpl.java:7235)

                oracle.jbo.server.ViewObjectImpl.activateIteratorState(ViewObjectImpl.java:18742)

                oracle.jbo.server.ApplicationModuleImpl.activateVOs(ApplicationModuleImpl.java:8172)

                oracle.jbo.server.ApplicationModuleImpl.doActivateState(ApplicationModuleImpl.java:7918)

                oracle.jbo.server.ApplicationModuleImpl.doActivateAMState(ApplicationModuleImpl.java:7884)

                oracle.jbo.server.Serializer.activate(Serializer.java:296)

                oracle.jbo.server.DBSerializer.activateRootAM(DBSerializer.java:330)

                oracle.jbo.server.ApplicationModuleImpl.activateState(ApplicationModuleImpl.java:6207)

 The looping was occuring in the getRow method of ViewRowSetImpl.  The stack trace also shows that the looping is occuring as the result of a activation event to restore a previously passivated application module.  This demonstrates the importance of testing code with the jbo.ampooling=false  setting in order to test for passivation problems.   In this case however we still might not have notice this problem because it requires both a large table and a specific sequence of user actions in order to reproduce.  

Here is the getRow method from ViewRowSetImpl.   The section in red was where the looping was occuring

   public Row getRow(Key key)

   {

      Row row;

      if (key != null)

      {

         Object keyHandle = key.getRowHandle();

         if (keyHandle != null && mQRef.getSignature() == key.getSignature())

         {

            row = getRowFromHandle(keyHandle);

            if (row != null && key.equals(row.getKey()))

            {

               return row;

            }

         }

      }

      //if there are rows in the collection, then match those before calling execute.

      if (getFetchedRowCount() > 0)

      {

         for (int j = 0; (row = getRow(j)) != null; j++)

         {

            if (key == null || row.getKey().equals(key))

            {

               return row;

            }

         }

      }

      //make sure that the query is executed before calling getRow(int);

      if (!isExecuted())

      {

         synchronized(getSyncLock())

         {

            execute(false /*force*/, false /*notify*/); //no need for eventing here.

         }

      }

     for (int j = 0; (row = getRow(j)) != null; j++)

      {

         if (key == null || row.getKey().equals(key))

         {

            return row;

         }

      }

      return null;

   }

Here is what I  did to duplicate the problem on my local jdeveloper instance.

• We have an entity based view object to a table that contains a large (2+ million) row set.  The view object was unbounded (no bind variables)
• The range size was set to 15. 
•  There is an iterator binding on the page bound to this view that is bound to a popup form for adding or inserting  one row of data. 
•  When the popup displays the unbounded query executes because the iterator executes when first used (the default setting).  This may not be good for database performance (which was taking ~3 seconds) but it only retrieves the 1st range of 15 rows from the database into memory and this did not normally cause a problem so was not noticed.
•  The data was entered on the form and a button “save and add another” is clicked which commits the current row and advances the form to a new row ready for input. 
•  At this point we have a view object that contains 1 range of rows (15) that were returned by the orignal query plus the newly inserted row that we just committed (16 rows in total).
•  Now to duplicate the problem we force a passivation of this users app module.  We did this by setting the pool size to 1 and opening up the same page in another browser window.
• We then cancel out of the form popup.  Before the form closes it presents an “Are you sure?” dialog and  it must re-activate the app module it is bound to.   During activation the listed getRow() method in ViewRowSetImpl is called in order to find and sync up all of the 16 rows that were in the view when it was passivated.  Since there was a newly inserted row it may be located near the end of the result set that would be returned by the unbounded query. (there was no sort order)   The loop in red is the problem.   The ADF framework was searching for this row by retrieving one row at a time from the database.  It basically had to go through all 2 million rows and pull them into memory before it would find it.  On my local debug machine this was happening extremely slow.  (About 50 per second) and I would run out of memory before it finished (and there was only 390,000 rows in our dev database)

This  could have been instantaneous if ADF would run a query with the ID to retrieve it, instead of searching for it row by row.  I really think this is a serious design flaw but the feedback from Oracle was that this was working as designed.   I don't understand why the activation process event needs to find the rows from the database.   It's purpose is to restore the state of the view object to what it was prior to passivation so why not just read the passivation record and be done with it?   This problem could occur on any large table that is used for both scrolling through the data and inserting rows using the same view object. 

Bottom Line and Lessons Learned.

1. If you have a view object that is only used for inserting rows, make sure that you set the Tuning parameter to Retrieve from the Database "No Rows" or "At Most One Row"  
2. If you do need to display a table that can filter and scroll over a very large result set and also insert new rows, use a read-only view object for displaying the data and a separate Entity based view object for inserting new rows.  
3. If the view object had had a sort order that caused newly inserted rows to appear at the beginning of the result set, we wouldn't have noticed this problem. 
4. This shows how critical it is to set the tuning parameters appropriately when dealing with large tables.   In our case this was a hidden time bomb waiting to happen.
   

Executing iterator binding in deferred Mode when there are required bind variable parameters

An iterator binding can be set to execute in "deferred" mode.  This is the default behavior if you don't change it.   What this means is that the query will not execute unless the page needs to display it's data.   This is great for use on a page that doesn't initially display the data to the user, like in a popup or tab that is not initially disclosed.  If the popup is displayed or the tab is disclosed, then and only then will the query be executed.  Also for tables that are set to lazy load (retrieve it's data after the page initially loads) this prevents the query from executing until the page is first displayed and the "Fetching data..."  message is displayed over the table.  A problem arises when the view object requires named bind variables.  The bind variables do not get set unless you add an ExecuteWithParams action binding to the task flow.   Calling an ExecuteWithParams will execute the query but that defeats the value of setting the iterator binding to "deferred"   So how do I set the bind variables without executing the query?

Here is one technique:

• Edit the View object, select the Java panel and check the "Generate View Object Class" and "Include bind_variable accessors.
• Edit the Client Interface and add the setMyValueBind()  method to the interface.
• Select the Data Control panel and click the refresh icon to refresh the data controls.
• Find your view object in the data controls panel and expand it.  You should see the setMyValueBind method under the list of attributes.
• Drag the method call over onto your task flow and add an EL expression to set the bind variables value.
• insert the method call into the chain of operations that execute prior to displaying the page.

That is all that is needed.  The first time the data is displayed, the iterator binding will execute the query using the set bind parameters and if the user never displays the popup or tab that displays the data, the query will not need to be executed.

How to cancel a long running query from the UI (part 1)

I've long wondered if there was any way to cancel a query that is taking too long to complete from the page where the query is being run, similar to hitting the cancel button in SQL Developer.   I noticed that the view object class has a cancelQuery() method on it but how do you call that when the thread is busy running the query.   The javadoc on the cancelQuery function says it can be called from another thread and this is what happens when you set a timeout value on the view object.   ADF spawns a monitor thread that will cancel the query after the timeout elapses.  I did some experimenting and found that if I overrode the executeQuery method in the view and saved the view object on the session I could then use that to cancel the query from another task flow running in another tab or browser window.
   Here is the code
    @Override
    public void executeQuery() {

        Map sessionScope = 
            ADFContext.getCurrent().getSessionScope();
        sessionScope.put("MyQuery",this);
        try{

        super.executeQuery();

        } finally{

          // only keep on the Session 
          //for the duration of the query  
          sessionScope.remove("MyQuery");

       }

On a different page put a button for canceling the query that calls this action method
  
public void cancelQueries(ActionEvent actionEvent) {
    Map sessionScope = ADFContext.getCurrent().getSessionScope();
    ViewObjectImpl vo = (ViewObjectImpl)sessionScope.get("MyQuery");
    if (vo!=null){
        if (vo.isDead()==false){
          vo.cancelQuery();        
        }
      sessionScope.remove("MyQuery");
    }
}
          
This will cause the query to throw a "JBO-27126 Long running query has been canceled" Exception.
In order to prevent the page from becoming broken you must catch this exception in the executeQueryForCollection method and re-throw a new JboException which will be displayed to the user.   See this blog for an example of that.


This seems to work fine, at least on 11G database.  According to this blog there my be problems making it work with the 10G drivers.  The next question is how would I put a button on the same browser tab as the one running the query.   I haven't implemented that but I can think of a couple of ways that might be done.  One would be to have the button call a javascript method that makes a custom AJAX call back to a plain old servlet that retrieves the users session and calls the cancelQuery method.  Another might be to add an inline frame that has a URL to a different task flow and page containing only the single button.  ADF doesn't let you submit a button on the same page until the page is no longer busy but using an inline frame is the same as running the task flow on another tab so concurrent requests could be made.