Blog do projektu Open Source JavaHotel

piątek, 27 grudnia 2013

InfoSphere Streams, REST API, Java application

Introduction
InfoSphere Streams contains API (application programming interface) which can be used to get access to different data related to Streams instance. The API is based on REST  and is implemented by using HTTP protocol. The format of data returned is JSON. More information is available here (chapter: REST API overview.
The API allows creating applications for monitoring and visualizing data related to the Strems instance and particular job or jobs. The Streams Console (provided with InforSphere Streams) is based  on REST API. More information is available here (chapter: Streams Console )
Authentication
The REST API is supported by SWS (Streams Web Services chapter: Streams Web Services). There is a difference between instance authentication (for instance: to use streamtool) and SWS authentication. There are two methods of SWS authentication: server (default) and client. More information is available here (chapter: Configuring security for the InfoSphere Streams REST API).
Server authentication
It is the default method. Firstly let start Streams Console application from any browser. Streams Console is based on REST API and allows to check if SWS is up and running and authentication is possible.
After starting the instance use the streamtool to get URL for Streams Console.
[streams@oc8442647460 ~]$ streamtool geturl -i test@streams

https://oc8442647460.ibm.com:8443/streams/console/login 

If login to Streams Console is successful we are sure that the access data is valid.
Sample Java code

 static class BusinessIntelligenceX509TrustManager implements
   X509TrustManager {

  public java.security.cert.X509Certificate[] getAcceptedIssuers() {
   return new java.security.cert.X509Certificate[] {};
  }

  public void checkClientTrusted(
    java.security.cert.X509Certificate[] certs, String authType) {
   // no-op
  }

  public void checkServerTrusted(
    java.security.cert.X509Certificate[] certs, String authType) {
   // no-op
  }

 }

 public static void auth1() {
  try {

   TrustManager[] trustAllCerts = new TrustManager[] { new BusinessIntelligenceX509TrustManager() };
   SSLContext sc;

   try {
    sc = SSLContext.getInstance("SSL");
   } catch (NoSuchAlgorithmException noSuchAlgorithmException) {
    return;
   }

   try {
    sc.init(null, trustAllCerts, new java.security.SecureRandom());
   } catch (KeyManagementException keyManagementException) {

    return;
   }

   HostnameVerifier hv = new HostnameVerifier() {
    public boolean verify(String urlHostName, SSLSession session) {
     return true;
    }
   };

   HttpsURLConnection
     .setDefaultSSLSocketFactory(sc.getSocketFactory());
   HttpsURLConnection.setDefaultHostnameVerifier(hv);

   // Retrieve the root resource information for Infosphere Streams
   URL url = new URL("https://st32:8443/streams/rest/resources");
   String userInfo = "streams:secret123";
   String authToken = "Basic "
     + DatatypeConverter.printBase64Binary(userInfo.getBytes());
   HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
   conn.setRequestProperty("Authorization", authToken);
   conn.setRequestMethod("GET");
   conn.connect();
   System.out.println("Response code: " + conn.getResponseCode());
   System.out.println("Content type: "
     + conn.getHeaderField("Content-Type"));
   String response = new BufferedReader(new InputStreamReader(
     conn.getInputStream())).readLine();
   System.out.println("Response: " + response);
   conn.disconnect();
  } catch (Exception e) {
   e.printStackTrace();
  }
 }


 public static void main(String[] args) {
  auth1();
 }
Client authentication 
If client authentication method is enabled only trusted clients can connect to SWS and REST API. Unfortunately, this method is a little bit more complicated and requires more preparation.
Enable client authentication 
It can be done from Streams Console. But the simplest method is to modify a file .streams/instances/{instanceid}/config/instance.properties and set SWS.enableClientAuthentication property to true.
Create and register client credentials
Create client credentials
Important: if the cn name (first and last name) is the same as instance owner then login name and the and password are not required during authentication.
 keytool -genkey -keyalg RSA -alias streams -storepass secret -validity 360 -keysize 1024 -storetype pkcs12 -keystore streamkey.p12
Add credentials to Web browser
keytool --export -keystore streamkey.p12 -alias streams -storetype pkcs12 -file my.crt
 For instance (Chromium): Properties -> Advanced -> Certificates -> Import certificate
Add certificate to SWS client keystore
streamtool addcertificate --clientid sb -f my.crt -i test@streams
Restart instance
streamtool stopinstance -i test@streams
streamtool startinstance -i test@streams
Streams Console
Open Streams Console again, should start without asking for credentials (if certificate first and last name is the same as instance owner, otherwise login and password is required)
Server certificate 
Next step is to create server trust store at the client site. Unfortunately, there is not visible method for getting server certificate using streamtool. The only solution I found (after Streams Console is successfully opened) is exporting server certificate directly from browser. In case of firefox : Preferences -> Advanced -> List certifacates -> Servers -> IBM branch. Export certificate as X.509 (DERT) and create server truststore.
keytool -import -alias streams -keystore servertrust -file serv.cert
Java sample 
Important: for some reason the client keystore create above does not work here. The only solution I found is to backup client certicate in Web browser (Firefox: Preferences ->Advanced -> List certificates -> Personal -> Backup as PKCS12) and used keystore created that way.
Important: This example assumes that client certificate contains cn (first and last name) which maps to instance owner or valid Streams user, so the authorization is ignored. Otherwise it is necessary to add basic authentication also (like example above).
 static void auth3() {
  try {
   // Identify locations of server truststore and client keystore
   System.setProperty("javax.net.ssl.trustStore",
     "/home/sbartkowski/servertrust");
   System.setProperty("javax.net.ssl.trustStorePassword", "secret");
   System.setProperty("javax.net.ssl.keyStore",
     "/home/sbartkowski/clientkeystore.p12");
   System.setProperty("javax.net.ssl.keyStorePassword", "secret");
   System.setProperty("javax.net.ssl.keyStoreType", "pkcs12");
   System.setProperty("javax.net.debug", "ssl");

   HostnameVerifier hv = new HostnameVerifier() {
    public boolean verify(String urlHostName, SSLSession session) {
     return true;
    }
   };

   HttpsURLConnection.setDefaultHostnameVerifier(hv);

   // Retrieve the root resource information for Infosphere Streams
   URL url = new URL("https://st32:8443/streams/rest/resources");
   HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
   conn.connect();
   System.out.println("Response code: " + conn.getResponseCode());
   System.out.println("Content type: "
     + conn.getHeaderField("Content-Type"));
   String response = new BufferedReader(new InputStreamReader(
     conn.getInputStream())).readLine();
   System.out.println("Response: " + response);
   conn.disconnect();
  } catch (Exception e) {
   e.printStackTrace();
  }
 }

niedziela, 22 grudnia 2013

JavaHotel - reservation panel improved


New version of JavaHotel has been uploaded (U/P user/user). The main difference is improved reservation panel : more compact, better navigation and better visualization of reservation.


The main challenge I fought with was adding colspan to CellTable. After spending a lot of time trying to apply different methods I ended up with copying and pasting DefaultCellTableBuilder and adding necessary changed to the underlying algorithm.

sobota, 7 grudnia 2013

IBM InfoSphere Streams, NetezzaLoad, ODBCAppend and input control operator

Introduction
In the previous post it was demonstrated how to connect to Netezza using connection data in the connections.xml file. But this method in not always convenient. For instance: assume we use different credentials in the test environment and production environment. Having credentials hardcoded in the connections.xml enforces recompiling our application every time it is deployed.
Solution
Happily there is a solution to this problem. It is possible also to send credentials during submit time and even to change them dynamically during a run time. It can be accomplished by applying "control" input stream to ODBCAppend and NetezzaLoad. Description (click on Operator Control Input Port). Unfortunately, there is no sample for this option and it is not obvious at the first glance how to use it.
Sample code
A sample code for connection using control input port is available here.

Remarks

  • The producer of connection data is Custom operator submitting credentials to both operators (ODBCAppend and NetezzaLoad). Although the credentials are hardcoded here it is not a problem to get them from somewhere, for instance a configuration file.
  • ODBCAppend and NetezzaLoad have a connectionPassword with invalid password hardcoded. It is on purpose to be sure that during a connection a password got from control port is used.
  • ODBCAppend and NetezzaLoad have parameter connectionPolicy set to Deferred (default is Immediate). Otherwise both operators would try to initialize connection at the beginning and fail. By means of Deferred value the connection is not started until first tuple arrives.
  • It is also necessary to postpone tuples arrival to ODBCAppend and NetezzaLoad until connection credentials are sent. Switch control serves as a gate between flow of tuples and both data operators. Switch operator having parameter start set to true serves as a gate latch. 
  • Custom operator sends credentials to ODBCAppend and NetezzaLoad and at the same time sends signal to Switch operator. The gate is broken and data can flow through not interrupted.
How it is visualized in the diagram above
  1. Beacon operator starts sending tuples. Tuples are sent (and blocked at the very beginning) to the FlowOfControl (Switch) operator.
  2. ControlPort (Custom) operator sends connection data to the ToODBC and NetezzaLoad operator. 
  3. After sending the connection data sends a signal to FlowOfControl (Switch) operator to open a gate.
  4. At the arrival of first tuple the connection is started using (valid) credentials from control input port (not from connections.xml file or invalid connectionPassword parameter).
  5. Then the constant flow of tuples from producer (Beacon) to consumers (ODBAppend and NetezzaLoad) is running. Tuples are passing through FlowOfControl gate without any friction.

sobota, 30 listopada 2013

Netezza Software Emulator, Linux and IBM InfoSphere Streams

Introduction
Netezza and InfoSphere Streams are the part of IBM Big Data offering. Although both are commercial products, good news is that for educational and non-production purpose also free versions are available.
Netezza Software Emulator can be downloaded here.
Non-production InfoSphere Streams package is available here. Although fresh installation of IBM InfoSphere Streams is not complicated it is a good idea to start with ready to use VMware image.
Netezza Software Emulator
Netezza in production requires dedicated hardware but for developing and discovering one can play with VMware machines (Host and SPU) behaving exactly like the real Netezza database. The Netezza Emulator is available only for Windows but it is possible to execute it also under Linux. But keep in mind that IBM supports only Windows version so in case of any problems you are left on your own.
Netezza Software Emulator for Linux
Download and execute INSEfDsetup.exe (it requires at least 13GB free disk space). After downloading transfer unpacked VMware machines (c:\Program Files\IBM\Netezza Software Emulator for Developer\VMS) to the Linux machine. Before starting them create additional vmnet3 network.

This network connects Host and SPU machine. Important: switch off DHCP and set IP address as 10.0.0.1.
Starting Netezza Emulator
Firstly start Host machine. After several minutes log in as nz user (standard password: nz) and execute nzstate. If output is like:
[nz@netezza ~]$ nzstate
System state is 'Discovering'.
[nz@netezza ~]$ nzhw
Description HW ID Location   Role   State
----------- ----- ---------- ------ -------
SPA         1001  spa1       None   Ok
Disk        1002  spa1.disk1 Active None
Disk        1003  spa1.disk2 Active None
Disk        1004  spa1.disk3 Active None
Disk        1005  spa1.disk4 Active None
Disk        1006  spa1.disk5 Spare  None
SPU         1007  spa1.spu1  Active Booting
[nz@netezza ~]$ 

In case of any problem simply launch nzstart command. Next step is to start SPU machine. The SPU machine should connect to Host and start networked booting. Do not bother about a good number of technical messages flying through the screen, it is as expected and does not mean that something wrong is happening. After some time execute again nzstate command in the Host machine. If the screen is like:
[nz@netezza ~]$ nzhw
Description HW ID Location   Role   State
----------- ----- ---------- ------ ------
SPA         1001  spa1       None   Ok
Disk        1002  spa1.disk1 Active OkSQL> create table testt (numb integer,name varchar(100))
SQLRowCount returns -1
SQL> select * from testt;
+------------+-----------------------------------------------------------------------------------------------------+
| NUMB       | NAME                                                                                                |
+------------+-----------------------------------------------------------------------------------------------------+
+------------+-----------------------------------------------------------------------------------------------------+
SQLRowCount returns 0

Disk        1003  spa1.disk2 Active Ok
Disk        1004  spa1.disk3 Active Ok
Disk        1005  spa1.disk4 Active Ok
Disk        1006  spa1.disk5 Spare  Ok
SPU         1007  spa1.spu1  Active Online
[nz@netezza ~]$ nzstate
System state is 'Online'.
[nz@netezza ~]$ 
it means that Netezza is started and ready to handle the requests. Then prepare a test database, a test database and a test user.
  nzsql
  create database test;
  create user testuser with password 'secret';
  grant all admin to testuser;
  grant all on test to testuser;
Check if testuser can log in into test database and execute some basic queries.
 nzsql test -U testuser -W secret
 create table x (x int);
 insert into x values(1);
 select * from x;
 drop table x;
Obtain Netezza connection software
This software can be downloaded from this place.Important: this software is available only for IBM customers and business partners.
Install and configure ODBC connection to Netezza on RedHat 6
Installing Netezza client is very simple and straightforward. Just unpack the Netezza client package (something like nz-linuxclient-v7.0.3-P2.tar.gz) and execute two unpack scripts, one in linux directory and the second in linux64 directory. The default installation location is /usr/local/nz.
Firstly test the connection by executing nzsqlodbc utility from /usr/local/nz/bin64 directory (netez is the host name of the HOST VMmachine in my environment).
[root@oc8442647460 bin64]# ./nzodbcsql -h netez test testuser secret

NZODBCSQL - program to test Netezza ODBC driver
            NOT FOR PRODUCTION USE


     Type 'quit' or '\q' or CTRL-D or ENTER at the prompt to quit
     NOTE: Max 100 rows are displayed (for selects)

 Driver version  : 'Release 7.0.3 (P-2) [Build 32506]'
 NPS version  : '07.00.0003 Release 7.0.3 (P-2) [Build 32506]'
 Database  : 'TEST'


nzodbc > 

Secondly configure unixODBC connection. That's very simple, just follow instructions in /usr/local/nz/lib64/ODBC_README note. In my environment I replaced host, database name, user name and password with netez,test,testuser and secret. Then test the connection using isql utility.
[sb@oc8442647460 ~]$ rlwrap isql NZSQL
+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+
SQL> create table x (x int);
SQLRowCount returns -1
SQL> insert into x values(1);
SQLRowCount returns 1
SQL> select * from x;
+------------+
| X          |
+------------+
| 1          |
+------------+
SQLRowCount returns 1
1 rows fetched
SQL> 
Important: I failed trying to configure unixODBC Netezza connection in Ubuntu (although nzodbcsql utility is working). Probably there are some incompatibilities with unixODBC version installed as default into Ubuntu.
Configure InfoShophere Streams environment to work with Netezza 
It is described in detail here.
It looks complicated but it is enough to add three entries to .bashrc file.

export STREAMS_ADAPTERS_ODBC_NETEZZA=NETEZZA
export STREAMS_ADAPTERS_ODBC_INCPATH=/usr/include/
export STREAMS_ADAPTERS_ODBC_LIBPATH=/usr/lib64/
Important: InfoSphere Streams can work with one database only. So if a connection with Netezza is specified the access to other database (like DB2) is not possible.
Create a simple InfoSphere Streams application to load data to Netezza
Source code (StremsStudio project) is available here. There are two standard methods to load data to Netezza.

  • Using ODBCAppend operator.
  • Using specialized NetezzaLoad operator.
Recommended method is the second one because using the ODBCAppend could create a significant performance bottleneck. The advantage of the first method is that by changing the ODBC data source name in the configuration file one can easy switch the application from one database to another. Firstly create a simple table to be populated by our sample.
SQL> create table testt (numb integer,name varchar(100))
SQLRowCount returns -1
SQL> select * from testt;
+------------+-----------------------------------------------------------------------------------------------------+
| NUMB       | NAME                                                                                                |
+------------+-----------------------------------------------------------------------------------------------------+
+------------+-----------------------------------------------------------------------------------------------------+
SQLRowCount returns 0

Both methods require preparing connection.xml file. The connection.xml file used in this example is available here.
Hint. While working with connection.xml I found pretty difficult to identify error in this file because the StreamsStudio is not  very talkative on this. But you can execute xmlint (XML syntax checker) directly from command line to get more explanatory output. Just copy and past command line invocation from StremsStudio console and remove redirection of error output to null device.


[sb@oc8442647460 NetezzaLoader]$ pwd
/home/sb/workspace/testp/NetezzaLoader
[sb@oc8442647460 NetezzaLoader]$ xmllint --noout --schema /opt/ibm/InfoSphereStreams/toolkits/com.ibm.streams.db/com.ibm.streams.db/Common/connection.xsd ./etc/connections.xml
./etc/connections.xml:29: parser error : Opening and ending tag mismatch: table line 23 and access_specification
    </access_specification>
                           ^
./etc/connections.xml:32: parser error : expected '>'
  </access_specifications>
                        ^
./etc/connections.xml:34: parser error : Opening and ending tag mismatch: access_specifications line 10 and st:connections
</st:connections> 
                 ^
./etc/connections.xml:34: parser error : Premature end of data in tag connections line 1
</st:connections> 
                  ^
[sb@oc8442647460 NetezzaLoader]$ 
The application code is available here. It is very simple. Beacon operator produces a series of randomly generated tuples. By modifying the iteration parameter one can increase the number of tuples generated. Removing this parameter for good produce the infinite sequence. Then the output is directed to ODBCAppend operator and and the same output  is directed NetezzaPrepareLoad and NetezzaLoad operator putting it to the Netezza database.
Next step is to launch application (as distributed), wait for the moment and verify the content of testt table.

--+
SQLRowCount returns 20
20 rows fetched
SQL> select * from testt;
+------------+-----------------------------------------------------------------------------------------------------+
| NUMB       | NAME                                                                                                |
+------------+-----------------------------------------------------------------------------------------------------+
| 31099      | I'm here                                                                                            |
| 35         | Good bye, cruel world                                                                               |
| 16515      | Good bye, cruel world                                                                               |
| 31099      | I'm here                                                                                            |
| 35         | Good bye, cruel world                                                                               |
| 16515      | Good bye, cruel world                                                                               |
| 5334       | Good bye, cruel world                                                                               |
| 1586       | Good bye, cruel world                                                                               |
| 5334       | Good bye, cruel world                                                                               |
| 1586       | Good bye, cruel world                                                                               |
| 14369      | Hello Kitty                                                                                         |
| 14369      | Hello Kitty                                                                                         |
| 8129       | Good bye, cruel world                                                                               |
| 4465       | Hello world                                                                                         |
| 8129       | Good bye, cruel world                                                                               |
| 4465       | Hello world                                                                                         |
| 15072      | I'm here                                                                                            |
| 20664      | Good bye, cruel world                                                                               |
| 15072      | I'm here                                                                                            |
| 20664      | Good bye, cruel world                                                                               |
+------------+-----------------------------------------------------------------------------------------------------+
SQLRowCount returns 20
20 rows fetched
SQL> 

niedziela, 24 listopada 2013

How to quickly create and remove Oracle database

Introduction
Sometimes it is necessary for me to quickly create an Oracle database, load data, perform some test and analysis and clear when it is done. I realized that it is much better to create a new database (instance) instead of constantly fleshing out the existing one. By removing this instance I can also reclaim disk space.
Create a database 
I want to create a tempdb database.
1. Prepare initialization parameter file.
tempdb.__oracle_base='/home/oracle/app/oracle'#ORACLE_BASE set from environment
*.db_create_file_dest='tempdb'
*.db_create_online_log_dest_1='tempdb'
*.db_name='tempdb'
*.db_recovery_file_dest='tempdb'
*.db_recovery_file_dest_size=10G
*.diagnostic_dest='tempdb'
2. Put this file in proper place (/home/oracle/app/oracle/product/11.2.0/dbhome_1/dbs/inittempdb.ora in my environment)
vi /home/oracle/app/oracle/product/11.2.0/dbhome_1/dbs/inittempdb.ora 
3. Make directory for instance data
mkdir /home/oracle/app/oracle/product/11.2.0/dbhome_1/dbs/tempdb 
4. Set ORACLE_SID
export ORACLE_SID=tempdb
5. Start sqlplus as DBA
sqlplus / as sysdba 
6. Create database, test user and run system scripts.
create spfile from pfile; 
startup nomount 
create database tempdb; 
create user test identified by secret;
grant all provileges to test;
@?/rdbms/admin/catalog.sql
@?/rdbms/admin/catproc.sql
@?/sqlplus/admin/pupbld.sql
shut
7. Prepare settings for remote access (file /home/oracle/app/oracle/product/11.2.0/dbhome_1/network/admin/tnsnames.ora) Add entries like:
LISTENER_TEMPDB =
  (ADDRESS = (PROTOCOL = TCP)(HOST = think)(PORT = 1521))

TEMPDB =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = sb-ThinkPad-T42)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = tempdb)
    )
  )
8. Add entry to /etc/oratab
tempdb:/home/oracle/app/oracle/product/11.2.0/dbhome_1:Y
9. Start database
dbstart
10. Prepare entry for sqldeveloper
11. Connect and act

Remove database
It is pretty easy (look also).

export ORACLE_SID=mydb
sqlplus / as sysdba
startup force mount
alter system enable restricted session;
drop database;
Remove also entries from /home/oracle/app/oracle/product/11.2.0/dbhome_1/network/admin/tnsnames.ora Reclaim disk space
rm -rf /home/oracle/app/oracle/product/11.2.0/dbhome_1/dbs/tempdb

piątek, 15 listopada 2013

Clojure, tautology and logical expression

Introduction
I decided to create a simple Clojure project for verifying logical tautologies. The source code is available here.

For instance:

 ((A \land B) \to C) \Leftrightarrow (A \to (B \to C)).
The algorithm is simple:
  1. Parse the logical expression
  2. Generate all possible combination of values for literals in the expression
  3. Evaluate the results for any value
  4. If every result is true then we have a tautology
It is not an optimal method for proving the tautology because the complexity is exponential but if we have a small number of literals it can run pretty fast.

Definition
Grammar for logical expression to be analyzed:

<expression> ::=
    [~] < expression >
         | <literal>
         | ( < expression > <operator> <expression>)

< literal > ::= chain of letter and digits starting from letter
< operator > ::= 
    & : logical AND
    ^ : logical OR
    == : logical equivalence
    => : logical implication
    ~ : logical NOT

Remarks:
  • I assume that every subexpression (except literals) should be enclosed in parenthesis. For instance: (A & B) but not (A & B ^ C). This I'm avoiding a problem related to operator precedence. Because any logical expression can be "normalized" that way it is not a limitation which matters.
  • The tautology quoted above can be put down that way: (( A & B) => C) == (A => (B => C))
  • One of De Morgan's Laws: ~(A & B) == (~A ^ ~B)
Parser
The first thing to do is the "tokenizer" - creates a flow of separated components. It can be achieved simply by means of regular expression.

  [seqw (re-seq #"[A-Za-z][\w]*|[\^\(\)&~]|=>|==" str)]

Source code for parser is available here.
 
defn parse
"parse string expression
 returns three element vector (Polish notation: (operator, left operand, right operand)
 left/right operand can be an atomic operand (variable) or 
 subexpression (another triada vector)"
[str]

Parser decomposes input string to "Polish notation". Every part of the expression is transformed to a triad [operator, left operand, right operand]. In the case of negation (one parameter operator) the right operand is nil.

Evaluation

The evaluation (counting value) is very simple while the expression is decomposed.

(defn evaluate 
    [expr val]

"evaluate (calculate value) of the parsed expression using current values (as hash-map)
 returns true or false
"
    (defn getval [subexpr]
    (let 
      [operator (first subexpr)
       first (fnext subexpr)
       second (nth subexpr 2)
       firstval (if (vector? first) (getval first) (get val first))
       secondval (if (vector? second) (getval second) (get val second))
      ] 
     (cond 
        (= operator nil) firstval
        (= operator "&") (and firstval secondval)
        (= operator "^") (or firstval secondval)
        (= operator "==") (= firstval secondval)
        (= operator "=>") (if (and firstval (not secondval)) false true)
        (= operator "~") (not firstval)
        :else nil
     )
     )
     )
     (getval expr)
)
Just scan the parsed expression and calculate. "Values" is the map: { "literal" : true/false }.
Verify the tautology
 By "verifying the tautology" I mean calculating the value for every combination of literals and making sure that it evaluates as "true" all the time. First thing is to extract all literals from the parsed string. It is received by a simple statement (assuming that 'e' is parsed expression)
distinct (filter #(and (not (nil? %)) (Character/isLetter  (first %))) (flatten e))
The number of all literal combination is 2 power {number of literals}. So generating all combinations can be achieved by iterating all integers from 0 to 2 power {number of literal} and extracting literal values from binary representation of the current integer (1 as true and 0 as false.
(defn genvalues [lvals num ]
     (defn getsinglevalue [k]
        [ (nth lvals k) (odd? (bit-shift-right num k)) ]
     )
     (map getsinglevalue (range 0 (count lvals)))
  )

The last but not least is the main "verify" function. The function returns a vector of values producing false. So the expression is tautology if the output list is empty.
(defn varifytautology

"verify if logical expression is tautology
 returns the collection of values giving false result
 if list is empty then expression is tautology
"

  [str]
  (let [ vals (calculateall str)]
;    (seq (for [x vals] (println x)))
    (reduce #( if (not (second %2 )) (conj %1 %2) %1) [] vals)
  )
)

Unit tests
The series of unit tests is provided with the source. Example
; DeMorgan's Low
(deftest parse-taut2
  (let [sent "(~(p & q) == (~p ^ ~q))"
        lfalse (logical.expression/varifytautology sent)
       ]
;   (println lfalse)   
    (is (empty? lfalse))
  )
)

; DeMorgan's Low
(deftest parse-taut3
  (let [sent "(~(p ^ q) == (~p & ~q))"
        lfalse (logical.expression/varifytautology sent)
       ]
;   (println lfalse)   
    (is (empty? lfalse))
  )
)

(deftest parse-taut4
  (let [sent "((a&b)=>c) == (a=>(b=>c))))"
        lfalse (logical.expression/varifytautology sent)
       ]
;   (println lfalse)   
    (is (empty? lfalse))
  )
)

; the Distribution Law
(deftest parse-taut5
  (let [sent "(p^(q&r)) == ((p^q)&(p^r))"
        lfalse (logical.expression/varifytautology sent)
       ]
;   (println lfalse)   
    (is (empty? lfalse))
  )
)
I'm happy to announce the De Morgan's Laws still holds true.

środa, 30 października 2013

Java properties and UTF-8

Introduction
There is an evergreen problem how to read Java properties from UTF-8 files. The source file should be ISO 8859-1 oriented and even overmighty Hercules cannot contend that.
Solution
But I found simply method which works for me. Just read a file in standard way, change all characters above 128 to Unicode escaped sequence and push through Properties.load method. Source file is available here.

public class ReadUTF8Properties { 

        private static String getFileContent(String name) throws IOException {
// does not work in Google App Engine, use Guava goodies            
//              return new String(Files.readAllBytes(Paths.get(name)));
            return Files.toString(new File(name),  Charsets.UTF_8);
        }

        private static String toLatin1(String s) {
                StringBuilder b = new StringBuilder();

                for (char c : s.toCharArray()) {
                        if (c >= 256 && c < 1000)
                                // 3 digits, add one leading 0
                                b.append("\\u0").append(Integer.toHexString(c));
                        else if (c >= 1000)
                                // 4 digits
                                b.append("\\u").append(Integer.toHexString(c));
                        else
                                b.append(c);
                }
                return b.toString();
        }

        public static Properties readProperties(String propName) throws IOException {

                Properties prop = new Properties();
                String p = toLatin1(getFileContent(propName));
                prop.load(new StringReader(p));
                return prop;
        }

}

wtorek, 15 października 2013

JavaHotel, first version

First version
I uploaded the first version of JavaHotel to Google App Engine. Test version is available: Java hotel for user (U/P user/user) and Java hotel for administrator (U/P admin/admin). Source files are available here. It is Google App Engine version but I'm planning to use it only for demo purpose. Also regular JEE version is created as a primary target. It will be available for Tomcat and Glassfish. Tomcat version is a single web application, Glassfish version is divided into persistence layer available as Session Stateless Bean and business logic layer. It was tested with Derby, Postgress and DB2.
It is the first version and everything is rough. But basic functionality is covered.
  • Prepare database: hotel, users, room, services, pricelist
  • Reservation panel
  • Booking
  • Check-in
  • Registering additional services during the stay
  • Billing
Reservation panel

Empty places denote free rooms, green booked and yellow with guests currently checked-in. Clicking at the empty place allows to make reservation starting from this day forward, clicking at the green allows to make check-in if customer just arrived and clicking at the yellow gives access to options related to stay (for instance: billing)
Reservation

It is the reservation dialog available after clicking at the empty room. User can modify number of days (default is one), calculate rating and enter the customer data. It is also possible to select the customer from the list of customers already registered.
Check-in
Check-in dialog pops up after clicking at the green (reservation) place. It allows transforming reservation to stay and starting billing the guest. In the check-in dialog it is also possible to register additional guests if the booking was done for more then one person.
Registering additional services

After stay has been started it is possible to append additional services to the stay. User can select a standard service prepared before or add a new services ad-hoc.
Billing
During and after the stay a bill can be issued. It is possible to issue more then one bill for one stay and every bill having a different payer. In the tab panel 'Not paid yet' page contains services not paid yet (without the bill issued for them) and 'All billable' contains the list of all services. The payment can be specified as 'Pay now' or 'Not paid now'. There is also possibility to register payment later for bills not paid now.
Conclusion
It is the first version so everything is basic and rough. But the most important functionality is implemented already. Next steps:

  • Booking of more then one room,  searching. For instance: implement scenario like: "Find the first available reservation for 5 persons: two double rooms and one single".
  • Issuing and printing the invoice.
  • Synchronization with calendar. Alerts when reservation or stay expires. 

niedziela, 22 września 2013

MVP framework, new features

Introduction
I added several enhancements to the MVP Jython framework. Source code is available here, sample application developed for Google App Engine can be launched using this web URL.
New types of main menu
So far only left side list of button has been available. Two new types of main menus have been added: "Up" menu and stack menu. "Up" menu is nothing more then image at the top status bar which expands after clicking giving a list of choices. Stack menu is an implementation of GWT StackPanel. "Up" menu can coexist with tab menu and stack menu but stack menu and tab menu excludes each otther. An example of stack menu is available here. More technical description.
"Up" menu

Stack menu














Image column
Image column contains icon or list of icons instead of text or number. Image list can be static or dynamic. In the second case a JavaScript code is executed to determine the list of images displayed depending on the row content. More detailed description. Icons plays also a role of buttons. After user clicks the icon a Jython server side action is raised. This way it is possible to implement an action depending on the row content and which icon has been clicked.

A running example is available here. Click "up" menu -> "List with icon". The up arrow is displayed for positive number and down arror for negative number. Left and right arrow is displayed depending if number is odd or even.
Different type for column and footer
As a default the footer type (and other properties like align and decimal position) are copied from column type. But it is possible to change this behavior.


A running example is available here. "Status" -> "List footer". The footer is under string column but the footer content is a number. Also align schema is different. More detailed description is available here.
Other changes
  • Dynamically modify the content of the status bar. The the status bar content can be customized depending on the context. More detailed description is available here. Running example can be launched. "Status" - > "Status text".
  • Load tab panel page on demand. An action is raised after clicking the page header. This way the tab panel page can be modified dynamically. More detailed description.
  • XML usage. XML format is often connected with love-hate relationship. But it is very convenient if data schema is not stable or can be changed. It is also very convenient if we want to pack a lot information in one entry without creating a table with tens or hundreds of columns. So I added some automation for packing and unpacking forms using XML file. More information is available here.  A running example (although nothing spectacular is visible) can be launched here. Click on "Dialog base xml".


środa, 28 sierpnia 2013

Clojure and polynomials

Introduction
I started reading through Clojure tutorial and found very interesting implementation of calculating polynomial and its first derivative. So I decided to stretch my math muscle and perform more polynomial arithmetic in Clojure basing on this article.
I'm assuming all the time that polynomial in Clojure is a list of coefficients from left to right (like in the written form). So list [2 8 9]  means 2x² + 8x + 9, [7 0 1 7] means: 7x³ + x + 7
Addition
Almost nothing more than (map + ...) function. The only problem is to expand shorter list (lower degree polynomial). Also improvement should be added to reduce the leading zeros from the result. Example:
(2x² + 8x + 9) + (-2x² + 2x + 4) = 0x² + 10x + 13

(defn addpol
  [pol1 pol2]
  ; degree of the sum polynomial 
  (let [nsize (max (count pol1) (count pol2) ) ]
  ; expand polynomial to the new degree by adding 0 coefficient at the frot
    (defn expandpol 
        [pol] 
            (concat ( repeat (max 0 (- nsize (count pol))) 0 ) pol )
    )
  )
  ; having both polynomials at the same degree just sum them
  (map + (expandpol pol1) (expandpol pol2))
)
Subtraction
Just use the previous function and negate the subtrahend.
(defn subtractpol
  [minuend subtrahend]
  ; negate subtrahend and run summing
  (addpol minuend ( map #(- 0 %) subtrahend))
)
Product
Multiply every coefficient of the first polynomial by every coefficient of the second polynomial. Recursion is used to iterate but this algorithm is a good candidate for loop->recur for better performance.

(defn mulpol
  [pol1 pol2]
  ( let
    ; level of the product
      [newlevel (- (+ (count pol1) (count pol2)) 1)]  
    ; multiply polynomial by digit and add zeros at the frond and end keeping it at the degree necessary
    (defn muldigit 
       [beg coeff end]
       (concat (repeat beg 0) (map #(* % coeff) pol2) (repeat end 0))
    )
    ; recursive as replacement for iteration
    (defn muladd
       ; 'beg' number of left zeros
       ; 'currentsum' list of coefficient multiplications peformed so far
       ; 'restpol1' list of coefficients not used yet
       ; 'end' number of right zeros
       [beg currentsum restpol1 end]
       (condp = restpol1
       ; end of recursion
       [] currentsum
       ; multiply and move to the next digit
       (
         map
            +
            currentsum
            (muladd 
               (+ beg 1)
               (muldigit beg (first restpol1) end)
               (rest restpol1)
               (- end 1)
            )
       )
       )
     )
  ; start of the recursion
  ( muladd 0 (repeat newlevel 0) pol1 (- newlevel (count pol2)))
  )
)
Division
Is more complicated because two results are expected: quotient and remainder. So to bring the result a map is used. Polynomial long division is implemented as easier. This function should be expanded to exclude division by zero (empty divisor) and reduce leading zeros from divisor. Example: (0 4 5)

; returns a map with two keys :quotient and :remainder
(defn divpol 
   [divident divisor]
   ; perform (lead(r)/lead(divisor)) * divisor
   (defn multerm
     [r resdiv]
       (concat (map #(* resdiv %) divisor) (repeat (- (count r) (count divisor)) 0))
   )
   ; recursive
   (defn div 
     [mapd]
   ( let [ q (get mapd :quotient)
           r (get mapd :remainder) 
           dv (/ (first r) (first divisor)) 
         ]
   ; end of recursion, pass down the result
   (if (or (< (count r) (count divisor)) (empty? r))
       mapd
   ; pull down the next digit from divident
       (div ( hash-map 
              :quotient (conj q dv)
              :remainder (rest (map - r (multerm r dv)))
            )
       )
   )
   )
   )   
   ; beginning of the recursion
   (div ( hash-map :quotient [], :remainder divident))
)   

niedziela, 18 sierpnia 2013

MVP Jython framework, html panel and javascript code

Introduction
The dialog is displayed using default layout. The default layout is convenient at the beginning but later probably something more useful and better looking is necessary. So it is possible to replace default layout by means of GWT HTMLPanel widget. Just add to the dialog definition file containing dialog layout in shape of html page and new display is visible. Also custom Javascript code can be added to the page.
TabPanel is very useful way of displaying a lot of data on the screen without scrolls. TabPanel can be built by combining html, css and javascript code but GWT contains TabPanel widget.
HTMLPanel and JavaScript custom code

Sample application is available here (click at "Dialog HTML panel"). The html, css and JavaScript code was downloaded from this page. It does not make a lot of sense here, it is only a presentation how default layout can be enriched with html, css and custom Javascript code. More detailed description is available here.
Adding HTMLPanel definition impacts only presentation part. It does not involve any change in backing Jython code.
TabPanel

Sample application is available here (click at "Dialog tab panel"). The usage of the TabPanel is limited currently. For instance it is not possible to disperse buttons between different tab pages. More detailed description is available here.  
Problems
  • Current implementation of HTMLPanel does not allow to use custom (internationalized) labels. It will be added later.
  • Current implementation of TabPanel is limited. More flexible approach will be added later.

czwartek, 1 sierpnia 2013

MVP Jython framework and edit grid

Introduction
I added a new widget to my MVP framework. It is an "edit grid" which allows modification data in the list directly (like in this GWT showcase) but allowing adding and removing rows dynamically.

This widget can be used when a number of fields is relatively small (containable in one line) and line editing is more natural then standard CRUD editing. Google App Engine live demo version is available here. Detailed description here, also source code and Selenium unit tests.
Problem
The main problem I found was to detect "the end of edit single line". There is no any direct "finish" button. This action is necessary to perform data validation and store data in persistence layer (assuming that one line reflects one record). The only solution I found was to discover the end of line editing after moving to the another line. In this case editing of the next line is suspended for moment and actions related to the last line are executed. If data validation fails then user is forced to return to the previous line and fix the data.
From that stems another problem. When user wants to finish editing he simply clicks "Accept" (or any other) button without moving to the next line. So it was necessary also to provide a way to declare which buttons requires validation of the last line.
 There are two ways of making data persisted. One is to persist data after any action relating to data change (after single column or whole line) or store all lines at the end of editing (after pressing "Accept" button or any other). In this case it is necessary to transport all data from the client to the server side. In order to avoid unnecessary traffic there is a way to specify which buttons trigger data transportation and which buttons do not. When data persisting is done at the end of editing there is a risk of losing all data just entered in case of any failure.
Next steps
I plan to use this widget in the JavaHotel application is several cases. The first case will be manually changing list price for reservation and bypass prices provided from regular price list.

wtorek, 30 lipca 2013

Byliśmy na przedstawieniu

20 lipca byliśmy na przedstawieniu filmu operowego "Kopciuszek" wyświetlonego w ramach 13 edycji Festiwalu "Ogrody Muzyczne" na Zamku Królewskim w Warszawie, podobało nam się bardzo.
Opera Rossiniego jest pozbawiona elementów bajkowych w porównaniu do wersji baśni Charles Perraulta, nie ma tutaj czarów dobrej wróżki. Autorzy przedstawienia postanowili jednak przypomnieć baśniowy charakter w postaci pięknych animacji pojawiających się w przerwach miedzy aktami opery.
Samo przedstawienie jest częścią szerszego projektu mającego na celu nagrywanie oper w historycznych wnętrzach z pełnym użyciem nowoczesnej techniki, tutaj za scenę posłużyły zabytkowe budowle z Turynu.
Samo przedstawienie nie ma jednego bohatera czy wiodącego solisty, tutaj wszystkie elementy są tak samo ważne i stanowią istotny element całości.
Autorzy nie zamierzali intrygować widzów jakimś rewolucyjnym i nowatorskim odczytaniem dzieła operowego, a raczej odwrotnie. Za pomocą nowoczesnej techniki wydobyć i pokazać to co najpiękniejsze, czyli wspaniałe wnętrza, piękną muzykę i postacie wykonawców, którzy nie tylko śpiewają swoje role ale także z wielkim talentem je odgrywają.
To zamierzenie udało się z całą pewnością. Mimo  iż cały film trwa blisko trzy godziny, to śledzi się od początku do końca z niesłabnącą przyjemnością i zainteresowaniem.

poniedziałek, 29 lipca 2013

Selenium, python and Firefox 22

Problem
Selenium refuses to work with Firefox22 although it works properly with Firefox21. A lot of posts poured over internet. Example :
https://groups.google.com/forum/#!topic/selenium-users/1psXxdH-zS0
Although there is a fix available for Selenium 2.22 it seems not working for python users.
My solution
By I find a way to force Selenium to use Firefox21 without removing Firefox22.
  • Download and install somewhere Firefox21 (for instance /home/opt/firefox21/firefox)
  • Force selenium starter to use it by adding browserConfigurationOptions parameter to start method. 
selenium = selenium(host, port, browser, http)
selenium.start(("executablePath=/home/opt/firefox21/firefox"))

niedziela, 30 czerwca 2013

MVP Jython framework and dateline widget

Introduction
I added "dateline" (not confuse with "timeline") widget to MVP framework. The demo version is available here (click "Panel date" button at the bottom). Source code for whole solution is available here. Sample screen snapshot.
This widget can be used as a dashboard for hotel application where lines are rooms and columns are rooms availability on timeline (free, reserved, in use).
Technical description is available here.
Features

  1. Buttons for navigating though time line (go to date, next day, previous day etc).
  2. Declaration of some visual behavior: number of lines visible, number of date columns.
  3. Lines customization. Dynamically created list of lines and definition of columns visible.
  4. Cell (line/date) customization. Dynamic definition of cell visualization via custom html. For instance: different visualization for free rooms and reserved rooms.
  5. Customized logic for clicking on cell. Possible to define some business logic to clicking. For instance: after clicking on the cell open a windows to start reservation of the room.
  6. Sorting (lines values only).

Future enhancement

  • Partial refresh. Current version calls for refresh any time something has changed. Optimization is possible, for instance: in case of moving one day forward/backward refresh only one column. 
  • The same for lines, request data only for visible line.
  • More convenient way for moving in time.
  • Customized searching.


poniedziałek, 24 czerwca 2013

Glassfish 4.0 and Guice

Problem
I was using a mixture of EJB3 and Guice injection following method described here. The sample source code is available here.
Glassfish 3.1
It works without problem with Glassfish 3.1 giving the desired result
Glassfish 4.0
But deployment failed when I tried the same code to push into Glassfish 4.0.
deploy?DEFAULT=/home/hotel/NetBeansProjects/Container/TestApp/dist/gfdeploy/TestApp&name=TestApp&force=true failed on GlassFish Server (1) 
 Error occurred during deployment: Exception while loading the app : CDI deployment failure:WELD-001408 Unsatisfied dependencies for type [InjectedService] with qualifiers [@Default] at injection point [[BackedAnnotatedField] @Inject private com.test.myejb.TestSessionBean.iService]. Please see server.log for more details.

It looks that CDI implementation in Glassfish 4.0 is trying to resolve @Inject annotation on its own just ignoring the fact that it should be handled by @Interceptors annotation.

@Stateless
@Interceptors(value = { GuiceInterceptor.class })
@Local
public class TestSessionBean implements TestSessionBeanLocal {

    // Add business logic below. (Right-click in editor and choose
    // "Insert Code > Add Business Method"
    @Inject
    private InjectedService iService;
    
    
    @Override
    public String getHello() {
        return "Hello, I'm your test session bean";
    }

    @Override
    public String getMessageFromInjected() {
        return iService.helloFromInjected();
    }


}
Solution 
Solution was quite simple, just replace:
//import com.google.inject.Inject;
import com.google.inject.Inject;
Another remedy was adding empty bean.xml file to META-INF directory.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://java.sun.com/xml/ns/javaee"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/beans_1_0.xsd">

niedziela, 23 czerwca 2013

Byliśmy na koncercie

5 czerwca 2013 roku byliśmy na koncercie organowym w ramach XXIII Międzynarodowego Festiwalu Muzyki Sakralnej, bardzo nam się podobało.
Uczeń Franciszka Lista, Julius Reubke, zmarł młodo i zostawił po sobie niewiele kompozycji. Ale jednak z nich, Sonata "Psalm 94", jest jednym z najchętniej wykonywanych utworów organowych epoki Romantyzmu.
Franciszek List rzadko tworzył muzykę na organy, tylko dwa większe utwory zyskały sobie większą popularność. Jeden z nich, Fantasie und Fuge über den Choral „Ad nos, ad salutarem undam”, był w repertuarze koncertu.
Pomimo, że nazwisko Franciszek List dużo więcej dzisiaj mówi niż Julius Reubke, to jednak pierwszy utwór bardziej zapadał w pamięć. List zdecydowanie bardziej się podoba jako autor muzyki fortepianowej czy symfonicznej, niż organowej.
Solistą na koncercie był Giampaolo Di Rosa, na bis wykonał bardzo szybki i efektowny utwór, ale nie udało nam się rozpoznać co to było.
Bardzo miłą zaletą koncertu w Katedrze Polowej Wojska Polskiego był telebim umieszczony w prezbiterium, który dawał możliwość obserwacji solisty w trakcie koncertu. Nie wszyscy sobie zdają sprawę, że muzyka organowa wymaga także obsługi klawiatury nożnej. Intensywne wykorzystanie pedałów widać było wyraźnie w utworze Juliusa Reubke. Wadą koncertów organowych jest dystans dzielący wykonawcę od publiczności, ale dzięki nowoczesnej technice ten dystans udało się zniwelować.

wtorek, 4 czerwca 2013

Number grid and JavaHotel application

Information
I added grid number to the JavaHotel application.





















It is used to define a price list for services offered by hotel (launch, U/P user/user -> Price list).
Next step
Hotel rooms, list of customers and dashboard to reserve/book and present hotel status

czwartek, 30 maja 2013

Jython MVP framework and new widget

Introduction
I extended "check list" (grid), also grid of numbers (not only checkbox) can be created. Demo version is available here, click "Number grid".
After clicking "Check" button the Jython backing action fills the bottom row with sum of numbers in the column.
More detailed information is available here.
Sample
Declaration of number grid is very simple, just add "type" info to  "checklist" tag. Example:

<checklist id="prices" displayname="Prices" type="decimal" afterdot="2" >
 </checklist>
Example of backing Jython code is available here.
Additional feature
Unlike "check box" number grid allows error visualization. In the sample application just enter number less then -10 and click "Check". More then one cell can be marked this way.
Test
Next test case has been added to the  test suite. Test scenario is available here.
Next step
This widget will be used in the hotel application to define a price list for services offered by the hotel.

MVP Jython framework and shiro authentication

Introduction
Next step in implementing Shiro authentication for JavaHotel application. Previously I developed an administrator module to add new users, hotels and to set permissions for each user in a particular hotel. Source code is available here.
This module is available in Google App Engine environment. English version and Polish version. User/Password : admin/admin. This module uses standard text based Shiro realm.
[main]
securityManager.sessionManager.sessionValidationSchedulerEnabled = false
[users]
admin=admin
[roles]
securityManager.sessionManager.sessionValidationSchedulerEnabled is disabled because it launches a thread to control session expire time but Google App Engine does not support threads.
Security implementation
After setting hotels and users next step is implemented. User can launch an application for a particular hotel and start activity for this hotel using credential implemented before. For the time being only one functionality is implemented (hotel services).
For accessing a particular hotel URL query hotel parameter is used. Example.
http://testjavahotel.appspot.com/?hotel=hotel
http://testjavahotel.appspot.com/?hotel=hotel1
Application uses another Shiro realm for this purpose.
Shiro realm
[main]
securityManager.sessionManager.sessionValidationSchedulerEnabled = false
hRealm=com.gwthotel.auth.HotelAuthRealm
inject=com.gwthotel.hotel.server.guice.HotelAuthResources
hRealm.iRes=$inject
Test
A simple test scenario can be performed.
  1. Launch http://testjavahotel.appspot.com/?start=admin.xml
  2. Add two hotels : hotel1 and hotel2
  3. Add two users: user1 and user2 (don't forget to set a password for them)
  4. Add user1 access to both hotels and and user2 only to hotel1.
  5. Launch http://testjavahotel.appspot.com/?hotel=hotel1
  6. Make sure that user1 and user2 can logon.
  7. Launch http://testjavahotel.appspot.com/?hotel=hotel2
  8. Make sure that user1 can logon and user2 cannot.
Future
Credentials setting should be modified. The solution that administrator is setting a password is invalid. Another method should be implemented, for instance:  sending a mail containing a temporary password or URL link to set the password for the first time or after resetting the password.

Problem
This application is using two different Shiro realm. During a test I found that I cannot use SecurityUtils.getSubject because after changing the realm a next call to getSubject retrieves users from the previous realm and I was unable to overcome it. The only solution I found was to give up SecurityUtils.getSubject and build Subject directly.
 private static Subject buildSubject() {
        Subject currentUser = new Subject.Builder().buildSubject();
        return currentUser;        
    }

    private Result authenticate(SessionEntry se, String tokenS) {
        SecurityManager securityManager = constructManager(se.getRealm());
        SecurityUtils.setSecurityManager(securityManager);
//        Subject currentUser = SecurityUtils.getSubject();
//        currentUser = new Subject.Builder().buildSubject();
        Subject currentUser = buildSubject();

niedziela, 26 maja 2013

Shiro, customized realm

Introduction
Shiro is very flexible and easy to extend security framework. But creating a custom realm is not  an easy task for the beginners. After some research I created a simple Shiro project to get the gist of the problem.
  1. Authentication using standard text based realm.
  2. The same using customized realm.
  3. The same using injected realm
The source code is available here (in shape of JUnit test case)
JUnit test case
The purpose is to run the same test case but using different realms. Source code.
 private void testShiro(String realM) {
        Factory factory = new IniSecurityManagerFactory(
                realM);
        org.apache.shiro.mgt.SecurityManager securityManager = factory
                .getInstance();
        SecurityUtils.setSecurityManager(securityManager);
        ...........
        test scenario
        ...........


    @Test
    public void test() {
        testShiro("classpath:shiro.ini");
    }

    @Test
    public void test1() {
        testShiro("classpath:custom.ini");
    }

    @Test
    public void test2() {
        testShiro("classpath:inject.ini");
    }

Important: Although test, test1 and test2 are included in one test suite every test should be performed independently. This is because the next test utilize the realm created in the previous test if run inside the same JVM. I did not find a simple way to overcome it.
Standard text based realm
shiro.ini
[users]
root = secret, admin
guest = secret, welcome
Customized realm
custom.ini
[main]
myRealm=com.custom.realm.MyRealm
com.custom.realm.MyRealm
Customized realm with injection
This example is a little more complicated because mini framework has been created. The "framework" contains custom realm and interface.

package com.custom.nextrealm;

import java.util.List;

public interface InjectCredentials {

    String getPerson();
    
    String getPassword();
    
    List getRoles();

}
The user can customize this customized realm by implementing this interface and providing login name, password and list of roles without bothering about all others details. The example of this customization for the purpose of the test :
inject.ini
[main]
myRealm=com.custom.nextrealm.MyRealm
inject=com.custominject.CustomCredentials
myRealm.iCrede=$inject
com.custominject.CustomCredentials

package com.custominject;
import java.util.ArrayList;
import java.util.List;

import com.custom.nextrealm.InjectCredentials;

public class CustomCredentials implements InjectCredentials {

    @Override
    public String getPerson() {
        return "guest";
    }

    @Override
    public String getPassword() {
        return "secret";
    }

    @Override
    public List<String> getRoles() {
        List<String> roles = new ArrayList<String>();
        roles.add("welcome");
        return roles;
    }

}

sobota, 11 maja 2013

Glassfish, EJB, access to

Introduction
I'm planning to prepare a modular version of my JavaHotel application having data access logic hidden in a stateless EJB but creating a simple EJB took me more time that I expected. On one hand it should be simple and other hand (particularly after reading this web page)  it is pretty complicated. So I created a simple Eclipse project showing all important points. Code is available here. The purpose is:

  • Create EJB stateless bean.
  • Access to this bean from standalone Java application to perform JUnit tests.
  • Access to this bean from another Web component.
  • Share common definition without duplicating code.
Shared code
I created Eclipse general (not Java) project being a placeholder for common definition. By virtue of Eclipse "Link source" option this code can be shared between other projects. This project contains above all the interface with business logic implemented by EJB. It is pure  Java interface without any EJB annotation stuff. This share code contains also ServiceLocator class resolving access to EJB.

package com.sample.ejbcalc;

public interface IHelloCalc {

    String getHello();

    int add(int elem1, int elem2);

}

package com.sample.ejblocator;

import javax.naming.InitialContext;
import javax.naming.NamingException;

import com.sample.ejbcalc.IEJBName;
import com.sample.ejbcalc.IHelloCalc;

public class EJBSampleLocator {
    
    public static IHelloCalc construct() throws NamingException {
        InitialContext ctx = new InitialContext();
        Object remoteObj = new InitialContext().lookup(IEJBName.JNDIRemote);
        IHelloCalc inter = (IHelloCalc) remoteObj;
        return inter;
    }

}

Bean implementation
Source code is available here.

package com.ejbcalc.impl;

import javax.ejb.EJB;
import javax.ejb.Remote;
import javax.ejb.Stateless;

import com.sample.ejbcalc.IEJBName;
import com.sample.ejbcalc.IHelloCalc;

/**
 * Session Bean implementation class CalcImpl
 */
@Stateless
@EJB(name = IEJBName.JNDIRemote, beanInterface = IHelloCalc.class)
@Remote
public class CalcImpl implements IHelloCalc {

    @Override
    public String getHello() {
        return "Hello, I'm your Calculator";
    }

    @Override
    public int add(int elem1, int elem2) {
        return elem1 + elem2;
    }

}
Pay attention to all  annotation stuff.
Standalone JUnit Java application
Source code is available here. Important (Glassfish). It is necessary to add gf-client.jar to classpath. This JUnit uses EJBSampleLocator utility from shared code.

import org.junit.BeforeClass;
import org.junit.Test;

import static org.junit.Assert.*;

import com.sample.ejbcalc.IHelloCalc;
import com.sample.ejblocator.EJBSampleLocator;


public class TestEJB {
    
    private static IHelloCalc i; 

    @BeforeClass
    public static void setUpBeforeClass() throws Exception {
        i = EJBSampleLocator.construct();
    }

    @Test
    public void test1() {
        assertEquals("Hello, I'm your Calculator",i.getHello());
    }

    @Test
    public void test4() {
        assertEquals(4,i.add(2, 2));
        assertFalse(i.add(2, 2) == 5);
        assertEquals(2,i.add(2, 0));
        assertEquals(2,i.add(0, 2));
        assertEquals(-1,i.add(2, -3));
    }

}
Web component accessing EJB
Last (but not least) Web component. Source code is available here. One servlet reuses common ServiceLocator (like standalone application). But it is also possible to use @EJB annotation and Glassfish  injects EJB directly.

package com.ejbjsp.servlet;

import java.io.IOException;

import javax.ejb.EJB;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.sample.ejbcalc.IHelloCalc;

@WebServlet("/Servlet2")
public class Servlet2 extends HttpServlet {

    private static final long serialVersionUID = 1L;

    @EJB
    private IHelloCalc i;

    protected void doGet(HttpServletRequest request,
            HttpServletResponse response) throws ServletException, IOException {

        response.getOutputStream().println("<h1>
 I'm Servlet 2 </h1&gt;
Bean greetings " + i.getHello() + "</pre&>
");
        response.getOutputStream().println("<br />
<h1>
 Ask Bean. What is 2 + 2 ? Bean's advice: " + i.add(2, 2) + " ! </h1>
");
        response.getOutputStream().flush();
    }

}

czwartek, 9 maja 2013

MVP framework, application and localization

Introduction
Several new features have been added to my MVP framework.
Localization
It is possible to create a fully localized application. Some localized message are framework embedded (for instance messages like: Yes, No, Accept, Resign) and it is possible also to create application specific messages.
More detailed description is available here.
Example of localized application (Google App  Engine).
English (default)
Polish
In both cases U/P is admin/admin
Check list widget utilization
In application available from above a check list widget is used in order to specify permission for the specific user in the specific hotel.
Development/production
The purpose is to develop Web application business logic only in Jython code without recompiling and redeploying the application. It means that during development every request should read fresh copy of jython code and dialog XML format. But of course it should be avoided in production code because it means significant reduction in performance. So development mode is introduced with the following features:
  1. Jython code, XML dialog and message bundles outside java class loader to avoid constantly republishing application to application server (when Eclipse project is open)
  2. No caching, just load resources directly form file system.
  3. Restart Jython with every request
  4. Every request takes a new copy of bundle messages.
Start of Java Hotel development
The application above is the first step in creating Java Hotel application. So far simple administration part is created - allows defining hotels, user and permissions.
Full source code of Java Hotel application created so far is available here. Also not Google App Engine (JPA based) version has been created.
Next step
Authentication to Java Hotel application using realm created in Administrative part described above.

środa, 1 maja 2013

Jython framework and new widget

Introduction
I added new widget to my MVP Jython framework. The purpose is to handle set of checking like that:


It is a matrix of logical values. It can be used like in the example above : to set permissions for a given user in the list of  hotels (or any other objects). The same user can have different permission in every hotel. Lines (rows) are hotels and columns are permissions. Of course - it is a matter of opinion if it were better to have hotels as lines and permissions as columns or opposite. The crux of the matter is to have the 'lines'/'columns' defined programmatically (dynamically) - not statically in the dialog definition.
Implementation
Sample implementation (Google App Engine application) is available here. Click "Check list" button at the bottom. Check/uncheck matrix elements and click "Check". The current values are sent to the Jython code in the background and two values in the upper half are set.
Defining check list in the dialog is very simple - just add checklist tag like in this sample. More than one check list is allowed. Handling values in the backing Jython code is unfortunately more complicated. We have to support three scenarios.
  • Defining lines and columns for the first time.
  • Setting values
  • Getting values.
A pretty complicated data structure is used to accomplish the task. It is described here. A sample Jython code is available here.
Future
  • It is possible to implement a matrix of number (and any other data types) but I can hardly find any use case which can implement it. 
  • Error messages applied directly to the matrix. For instance message like "Checking here is not allowed" in the direct matrix position.
Next steps
I will try to implement the beginning of my new hotel management application. It will be administrative part: defining of hotels, user and permission of user in the particular hotel.  Then implement this solution as a realm for 'Shiro' to implement security for a hotel management (not administrative).

niedziela, 21 kwietnia 2013

New version of JSPWiki in the clouds

Problem
I realized that my humble JSPWiki implementation for Google App Engine is under spam attack. Nasty anonymous user is constantly adding spam message to the main page. I also realized that ACL (Access Control List for pages) does not work in my implementation so I was unable to restrict access to this page.
Solution
After spending some time I enabled ACL for page access. So one can add to the pages stuff like [{ALLOW view SocialCommittee}] and the access is limited. The problem was awkward binding ACL list with the 'WikiPage' class I found in the standard version. But now it seems working, I restricted access to the 'Main' page and although the page was under spam attach all the time the spam message was not added. I also put the spam address to the blacklist (using solution described here) and the battle with spam is over and victorious for the time being.