New version of JSPWiki in the clouds

Problem
I realized that my humble JSPWiki implementation for Google App Engine is under spam attack. Nasty anonymous user is constantly adding spam message to the main page. I also realized that ACL (Access Control List for pages) does not work in my implementation so I was unable to restrict access to this page.
Solution
After spending some time I enabled ACL for page access. So one can add to the pages stuff like [{ALLOW view SocialCommittee}] and the access is limited. The problem was awkward binding ACL list with the 'WikiPage' class I found in the standard version. But now it seems working, I restricted access to the 'Main' page and although the page was under spam attach all the time the spam message was not added. I also put the spam address to the blacklist (using solution described here) and the battle with spam is over and victorious for the time being.

Jython framework and security

Introduction
I added security to my Jython/UI MVP framework. Full source code is available here. Current Google App Engine version is available here. Another version of an application (without security enabled) is available here. Also not Google App Engine (tested with Tomcat and Glassfish) has been created. Current documentation is available here.
Authentication and Shiro
Authentication is implemented using Apache Shiro. Only standalone API is used now. I found this framework very useful - it fulfills what is needed and is very simple at the same time. In sample application the simplest authentication realm is used but the doors is open to implement more sophisticated authentication environment.
Authentication and two modes
It is controlled by Authenticate property in app.properties file (example).

• Authenticate=Y : Restricted mode, access to all pages requires authentication token. Cannot get access without being authenticated.
• Authenticate=N (default) : Not restricted mode. Access for not authenticated users to some pages is possible.

Login enforcement

Starting pages (specified by 'start' http query argument) enforcing login page are specified in app.properties file. Parameter 'Login' should contain list of starting page requiring login window. All starting pages not specified there are accessible without authentication (only if not restricted mode is enabled).

Authentication token

After successful login authentication token is returned to the client. From now on all access to the server  should be performed by using this token. Any password (with except to the login page) is stored at the client side just making all stuff more secure.

Authentication token and server

Unfortunately, this solution does not come cheap. It is related to the fact that the purpose is to have server side stateless, just ready to run in cloud environment. So the solution cannot rely on Shiro session  because next request can be executed in a fresh instance of JVM just having previous session invalidated.  The problem is described here in Shiro documentation. So both user name and password should be stored at the server side after user authentication ready to authenticate again against the Shiro session. Credentials are stored in a cache and also in backing persistent storage in case of being removed from cache. Implementation of cache and persistent storage is different for Google App Engine and not Google App Engine implementation. The simple algorithm for caching credentials is common.

public class SecuritySessionStore implements ISecuritySessionCache {

    private final ISecuritySessionMemCache iMemCache;
    private final ISecuritySessionPersistent iPersistent;

    private static final Logger log = Logger
            .getLogger(SecuritySessionStore.class.getName());

    @Inject
    public SecuritySessionStore(ISecuritySessionMemCache iMemCache,
            ISecuritySessionPersistent iPersistent) {
        this.iMemCache = iMemCache;
        this.iPersistent = iPersistent;
    }

    @Override
    public Object get(String key) throws InvalidClassException {
        Object val = iMemCache.get(key);
        if (val != null)
            return val;
        val = iPersistent.get(key);
        if (val == null)
            return null;
        // add value to cache again
        log.log(Level.FINE, LogMess.getMessN(ILogMess.PUTINCACHEAGAIN, key));
        iMemCache.put(key, val);
        return val;
    }

    @Override
    public void put(String key, Object o) {
        iMemCache.put(key, o);
        iPersistent.put(key, o);
    }

    @Override
    public void remove(String key) {
        iMemCache.remove(key);
        iPersistent.remove(key);
    }

}

By virtue of Guice the proper implementation of ISecuritySessionMemCache and ISecuritySessionPersitent is injected.
Authorization and Jexl
Three types of authorization is used. User, role and permission. Sometimes authorization rule can be complicated (for instance: give access to the user identified by name or the user having the role). So I decided to user Jexl for creating boolean expression evaluating to True (is authorized) or False (is not authorized).
To create an expression three methods are defined: sec.u (current user is the user authorized), sec.r (current user has the role) and sec.p (current user has permission). Using this three methods even complicated expression can be created.
Example:
Only 'darkhelmet' user is authorized and all user belonging to 'shwartz' role. Jexl expression: sec.u('dearkhelmer) or sec.r('schwartz')
Two types of authorization
Static way, restrict access in the xml dialog definition. Two attributes : 'readonly' and 'hidden' can be enriched with security expression. If expression evaluates to True for current user then attribute is enabled otherwise is disabled. This attributes can be attached to fields in the form, columns in the list and buttons. If 'hidden' is enabled then simple the element is not visible and not accessible. If 'readonly' is enabled then the element is visible but cannot be modified or clicked (in case of button).
Dynamic way, from 'jython' code. Authorization can be also enforced in 'jython' code at the server side just allowing to apply any logic to authorization rule. Examples:

from guice import ServiceInjector

def dialogaction(action,var) :
  iSec = ServiceInjector.constructSecurity()
  token = var["SECURITY_TOKEN"]
  ok = iSec.isAuthorized(token,"sec.u('darkhelmet')")
  var["OK"] = ok

Tests
 Both types of tests: GUI testing using Boa/Selenium framework (test cases) and Junit test cases have been created for security enhancement.
Future, next steps
It is the first version of security enhancement. Further development:

• 'Hidden'. Current version simple applies 'hidden' attribute to HTML element. It is not rendered but is visible in HTML code. Future version will contain 'superhidden' attribute. Such a field will not be included in the HTML code.
• Current version stored password at the server side in plain text. It should be encoded in real production code.
• More attributes covered with authorization.

Shiro and JdbcRealm

Introduction
I started playing with Shiro security framework but soon got a little upset that it was not easy to learn how to setup a jdbcrealm. It took me some time to configure such a realm in the simplest possible way without introducing any new instances of additional objects. Warning: this solution is not safe because it stores credentials unsecured and should be used only for evaluation.
So I created a simple java code which allows to execute Shiro introductional tutorial and achieve the same result.
Solution (using Derby database)
shiro.ini (pay attention to the last line necessary to recognize permissions)

[main]
jdbcRealm=org.apache.shiro.realm.jdbc.JdbcRealm
 
ds = org.apache.derby.jdbc.EmbeddedDataSource
ds.user = APP
ds.password = APP
ds.connectionAttributes=databaseName=nameofDatabase
ds.databaseName=/tmp/realm;create=true
jdbcRealm.dataSource=$ds

;cacheManager=org.apache.shiro.cache.ehcache.EhCacheManager 
;cacheManager.cacheManagerConfigFile=classpath:ehcache.xml
;securityManager.cacheManager=$cacheManager

jdbcRealm.permissionsLookupEnabled=true

Java code

  private static String[] dropSchema = { "DROP TABLE USERS",
            "DROP TABLE USER_ROLES", "DROP TABLE ROLES_PERMISSIONS" };

    private static String[] createSchema = {
            "CREATE TABLE USERS (USERNAME VARCHAR(128), PASSWORD VARCHAR(128))",
            "CREATE TABLE USER_ROLES (USERNAME VARCHAR(128), ROLE_NAME VARCHAR(128))",
            "CREATE TABLE ROLES_PERMISSIONS (ROLE_NAME VARCHAR(128), PERMISSION VARCHAR(128))" };

    private static String[] insertData = {
            "INSERT INTO USERS VALUES('root','secret')",
            "INSERT INTO USERS VALUES('presidentskroob','12345')",
            "INSERT INTO USERS VALUES('darkhelmet','ludicrousspeed')",
            "INSERT INTO USERS VALUES('lonestarr','vespa')",
            "INSERT INTO USER_ROLES VALUES('root','admin')",
            "INSERT INTO USER_ROLES VALUES('presidentskroob', 'president')",
            "INSERT INTO USER_ROLES VALUES('darkhelmet','darklord')",
            "INSERT INTO USER_ROLES VALUES('darkhelmet','schwartz')",
            "INSERT INTO USER_ROLES VALUES('lonestarr','goodguy')",
            "INSERT INTO USER_ROLES VALUES('lonestarr','schwartz')",
            "INSERT INTO ROLES_PERMISSIONS VALUES('admin','*')",
            "INSERT INTO ROLES_PERMISSIONS VALUES('schwartz','lightsaber:*')",
            "INSERT INTO ROLES_PERMISSIONS VALUES('goodguy','winnebago:drive:eagle5')" };

    private static void executeSQL(Connection con, String[] sql)
            throws SQLException {
        for (String s : sql) {
            con.createStatement().execute(s);
        }
    }

    private static void executeSQLE(Connection con, String[] sql) {
        for (String s : sql) {
            try {
                con.createStatement().execute(s);
            } catch (SQLException e) {
                continue;
            }
        }
    }

    private static void createDB() {
        EmbeddedDataSource ds = new EmbeddedDataSource();
        ds.setDatabaseName("/tmp/realm;create=true");
        ds.setUser("APP");
        ds.setPassword("APP");
        Connection con = null;
        try {
            con = ds.getConnection();
            executeSQLE(con, dropSchema);
            executeSQL(con, createSchema);
            executeSQL(con, insertData);
            con.close();
        } catch (SQLException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

    /**
     * @param args
     */
    public static void main(String[] args) {
        createDB();

        // The easiest way to create a Shiro SecurityManager with configured
        // realms, users, roles and permissions is to use the simple INI config.
        // We'll do that by using a factory that can ingest a .ini file and
        // return a SecurityManager instance:

        // Use the shiro.ini file at the root of the classpath
        // (file: and url: prefixes load from files and urls respectively):


.................. rest of the tutorial code ..............

Byliśmy na koncerce

Dnia 21 marca 2013 byliśmy na koncercie muzyki Antona Brucknera w ramach XVII Festiwalu Wielkanocnego Ludwiga van Beethovena. Uczucia jednak mieliśmy mieszane. Te Deum, które zostało wykonane na początek, jest uważane za najważniejsze religijne dzieło muzyczne tego kompozytora. Niestety, ale nie dane nam było to uznać. Albo kompozycja jest za duża, albo sala Filharmonii Narodowej za mała. Potężne dźwięki chóru, organów, orkiestry z dominującą rolą instrumentów dętych i perkusji zdawały się rozsadzać scenę i widownię. Można było tylko współczuć słuchaczom siedzącym w pierwszych rzędach. Chór usiłował przekrzyczeć orkiestrę, a orkiestra starała się do tego nie dopuścić. A już zupełnie nie udawało się przebić solistom,w zasadzie wykonanie mogłoby się bez nich obyć.

Na szczęście główną atrakcją wieczoru była IV Symfonia i dla tego wykonania warto było kupić bilety. IV Symfonia (Romantyczna) to dzieło bardzo popularne i chętnie wykonywane, nie tylko w dorobku Brucknera ale całej muzyki dawnej. Jednak nigdy nie było nam dane słuchać tej kompozycji na żywo, a w wypadku tego utworu nawet najbardziej doskonałe nagranie nie może się jednak równać z salą koncertową. Tutaj na szczęście partii organów i chóru nie ma, zaś dyrygentowi udało się zachować równowagę między instrumentami dętymi i pozostałą częścią orkiestry. To także dzieło potężne i mocne, miejscami przytłaczające, ale wszystkie proporcje i brzmienia były utrzymane w ryzach. Słuchacze po zakończeniu zasłużenie oklaskiwali orkiestrę, a zwłaszcza muzyków grających na instrumentach dętych. Byli na pewno głównymi bohaterami tego wieczoru i bardzo słusznie dyrygent podnosił ich jako pierwszych do oklasków.

MVC (MVP) framework, the show must go on

MVP framewok
I created the next version of Jython/GWT MVC framework. But I would rather call it following MVP (not MVC) framework. Current demo version (Google App Engine implementation) is available here. The demo was also tested with Tomcat and Glassfish. Prerequisite: Derby database (should be available somewhere in tha class path). Instruction how to setup Eclipse project is available here. Pom.xml file (maven) will be provided later. Current state of art is described here.

General description
The general structure is presented below :

• VIEW part. Based on GWT. Consisting of two packages. GWT UI and Jython UI. Creates UI components and interacts with PRESENTER via server Java code.
• DTO (Data Transient Objects). Transfers data between client (browser) side and server side. It is simple Java maps and lists of maps (as rows). Java shared (shared between GWT and server code) code (the same for application data and XML metadata) is available here.
• Server side Java code. Receiver of GWT RPC calls. Adapter between client (browser) side and PRESENTER Jython code. Transform Java maps and list to Jython data structures (dictionary and sequences) and vice versa. Source code.
• MODEL. Jython dictionaries and sequences of dictionaries (rows). Carriers of data between application code and VIEW part. 
• PRESENTER. Jython code, application logic. Responses to the client action, sends data to background database and extracts data. Sample application code.
• XML metadata. Is used by VIEW to create UI widgets and objects. Sample data is available here.

Features implemented so far

• Reading lists in pages (chunks). It is necessary for big lists (of course - it depends what one means by "big").
• Alert and error windows messages.
• Simple data validation at the client side (more complicated validation should be performed at the application-Jython code).
• Additional UI widgets: date and time picker, rich text format, mulitline editing.
• Confirmation for add/delete/change CRUD actions.
• Custom helpers, also "select" tag.

Testing

GUI testing (additionally to JUnit tests). GUI testing is performed by Selenium extension to BoaTester. The 'selenium' best test cases are available here. 

Next step

Next step is to implement security: authorization and authentication. I plan to use Apache Shiro. It is simple and seems to cover all topic necessary here.

BoaTester, new version

Introduction
I deployed new version of boatester framework. New features include changes related to 'selenium' base test cases. New source code is available here.

1. Tested with Selenium 2.
2. Allows launching 'selenium' test cases automatically. Previously every 'selenium' test cases required Python starter. Now it is not necessary nevertheless 'Python' started method (more flexible) is still possible and can be mingled with automatic tests.
3. 'call' action. Run another piece of code. It allows code reusing. For instance: very often the same piece of code is executed at the beginning like: authentication or waiting for objects to be created. 
4. 'select' action. Related to 'select' tag. For reason I do not understand 'click' does not work here. So additional command was necessary.
5. 'waitForNot' action. Opposite to 'waitFor'. Waits until object disappears from the screen. 

Future

Enhancement for 'call' action. Add parameters like regular procedures in programming languages. For instance: there is a more complicated piece of code to check some value. The 'value' to be checked can be a passed as a parameter just allowing using this code in different contexts.

DB2 Task Scheduler and web UI

Introduction
DB2 Task Scheduler allows to run and maintain administrative tasks executed after the predefined schedule. "Administrative task" is nothing more then SQL Stored Procedure designed to do some job. Task scheduler is not enabled by default, in order to activate it one has to configure it manually. DB2 task scheduler is similar to Oracle DBMS_JOB package. Unlike to DMBS_JOB it does not allow to execute SQL code directly, stored procedure should be created before.
There are several DB2 routines which allows to define new task, modify or remove existing task and monitor the current task status. Unfortunately, everything should be done manually. There is no IBM tool which provides  GUI to make this task less painful. Neither IBM Data Studio nor   IBM Data Studio web console do not have any windows or view giving access to task schedule. Job manager available in IBM Data Studio web console is a different solution, it has nothing in common with native DB2 task scheduler.
So I decided to fill the gap and create my own web application covering the DB2 task scheduler.
Installation
Read readme.txt, install maven, copy and paste pom.xml file, run:

maven compile

ignore all warnings and deploy target/db2jobscheduler.war.
After launching application the following screen should appear.

The application was tested with Tomcat 7 and Glassfish.
Project
It is an open source project created by virtue of MVC JythonUI framework. This framework has been significantly enhanced ever since.
Java source code (minimalistic) is available here. XML files defining user interface are available here. The most important part, Jython scripts containing all business logic, is available here. zxJDBC  package was used for accessing the DB2 database.
Source code structure
Java code serves only as a glue for initializing and starting the application. The development is focused on XML file describing the UI and Jython script file for business logic. It was the purpose of the JythonUI framework.

Functionality	XML files (UI)	Jython files (business logic)
Main menu	start.xml
Adding and modifying single source list	datasource.xml	datasourcelist.py
Data source list	connectiondef.xml	datasourcelist.py
List of tasks	tasklist.xml	shed.py
Adding and modifying single task	taskdescr.xml	shed.py
Current task status	taskstatus.xml	taskstatuslist.py
Helper for displaying a list of SP in the schema	proclist.xml	proclist.py
Helper for list of schemas	typesenum.xml	typedef.py
Helper encapsulating access to database		db2action.py
Helper encapsulating access to datasource definition		datasource.py

Functionality implemented

• Defining data source (s).
• List of tasks already defined. ADMIN_TASK_LIST
• Add new task. ADMIN_ADD_TASK
• Modifying existing task. ADMIN_TASK_UPDATE
• Remove task. ADMIN_TASK_REMOVE
• Task status. ADMIN_TASK_STATUS

Datasource defining

Displays list of datasource available now. Allows adding, modifying and removing datasource definition.

Administrative tasks maintaining

Displays list of scheduled task. Allows adding, modifying and removing tasks.

Task status

Current status of the tasks. 

List is displayed in chunks (there could be thousands of tasks executed already). List can be filtered (display a subset only) and sorted by any attribute.

Current status of the project

It is the first version with simplistic user interface. But all functionality of DB2 task scheduler is implemented.

Future development

• Improve user interface, help texts, direct connection to appropriate entries in DB2 Infocenter. 
• Security, user authentication.
• Encode database credentials, now password is saved as plain text.
• User authorization, only authorized user can add new task or modify existing.
• The same for datasource definition.
• Make task handling more convenient for the user. "Task" in DB2 is a stored procedure. Current implementation requires defining administrative stored procedure outside application (for instance by  using IBM Data Studio) then enacting it in the task scheduler application. Add simple text editor for creating and modifying stored procedure body in the application and deploying it to the database server without leaving the application.
• More convenient way for defining the task schedule. 
• Keep and maintain task schedule off-line repository. Additional database (for instance in the shape of XML files) containing administrative stored procedure body and task header for later reuse.
• Copy and paste tasks between databases - for instance copy task definition from test database to production database.
• More convenient way of disabling/enabling tasks.
• Export and import of administrative tasks. Task header and stored procedure body.