Blog do projektu Open Source JavaHotel

niedziela, 29 marca 2015

OpenLDAP, Ubuntu, TLS

Problem
I installed OpenLDAP on my Ubuntu 14.04 machine. It worked fine until I tried to authenticate from RHEL 6.6 box. RHEL 6.6 LDAP client requires TLS connection and there is not way to persuade it to change its mind. So I enabled OpenLDAP server for TLS connection using advice provided here. But in turn OpenLDAP server refused to restart. In /var/log/syslog file I found enigmatic entry:

Mar 29 00:47:57 sb-ThinkPad-W540 slapd[11071]: connections_destroy: nothing to destroy.
Mar 29 00:47:57 sb-ThinkPad-W540 kernel: [14766.527083] type=1400 audit(1427586477.983:83): 
apparmor="DENIED" operation="open" profile="/usr/sbin/slapd" name="/usr/share/p11-kit/modules/" 
pid=11070 comm="slapd" requested_mask="r" denied_mask="r" fsuid=122 ouid=0
Solution
After browsing I discovered that there exists linux kernel guardian called AppArmor and this Cerberus denies OpenLDAP server access to some configuration directories. But the solution was quite simple. Adding several lines to slapd profile opens this gate and now my LDAP server works as expected
File /etc/apparmor.d/usr.sbin.slapd
/usr/share/p11-kit/modules/ r,
/usr/share/p11-kit/modules/* r,
/usr/lib/x86_64-linux-gnu/pkcs11/ m,
/usr/lib/x86_64-linux-gnu/pkcs11/* m,

piątek, 13 lutego 2015

SQL Server to DB2 migration, UDF

Introduction
One of the element of any migration is built-in functions conversion. Some of them are the same, some are similar but different and some does not have any direct equivalent and requires additional developing. Freely available http://www.redbooks.ibm.com/redbooks/pdfs/sg246672.pdf ("Function mapping" chapter) contains a good number of examples how to convert SQL Server built-in functions to DB2.
Conversion is quite easy but what to do if we want to get SQL statements migrated to DB2 and keep backward compatibility at the same time.
It is more complicated but also possible. I created a simple project with examples - it is available here.
Solution for DateAdd, DateDiff and DatePart
The main problem is DateAdd and DateDiff functions. Example:
SELECT DATEADD(month, 1, '2006-08-30');
Unfortunately, this statement cannot be migrated directly to DB2 because DB2 does not support enumeration type. So it is necessary to modify also SQL Server statement to achieve backward compatibility.
Instead of using DATEADD(month ...) function replace it with DATEADD_MM function.
CREATE FUNCTION DATEADD_MM(@mins INT, @da DATETIME)
RETURNS DATETIME
AS 
BEGIN
  RETURN DATEADD(mm,@mins,@da)
END 
and modify the statement (all not built-in UDF in SQL Server should be qualified with schema name).
SELECT dbo.DATEADD_MM(1, '2006-08-30');
DB2 equivalent
CREATE OR REPLACE FUNCTION DBO.DATEADD_MM(IN NOD INT, IN DAT TIMESTAMP)
RETURNS TIMESTAMP
RETURN DAT + NOD MONTHS
@
and DB2 SQL stamement
SELECT dbo.DATEADD_MM(1, '2006-08-30') FROM SYSIBM.DUAL;
After this transformation function dbo.DATEADD_MM can be used any way in the SQL statements executable in SQL Server and DB2 without any modification.
In the files:  DB2 and SQL Server are more examples for DATEADD, DATEDIFF and DATEPART function.
Solution for other built-in scalar functions
Other functions are easy to migrate, just create its DB2 equivalent.
Example:
SELECT SUBSTRING('ABCDEF',2,2)
DB2 equivalence
CREATE OR REPLACE FUNCTION SUBSTRING (IN STR VARCHAR(32672),IN STA INT, IN LEN INT)
RETURNS VARCHAR(32672)
  RETURN SUBSTRING(STR,STA,LEN,CODEUNITS16)
@
More examples:  DB2 and SQL Server
Performance issue
A question can be raised if the performance will not suffer. The question is worth considering because instead of replacing SQL Server function by DB2 equivalent we are adding additional wrapper function to keep backward compatibility. Using additional UDF (particularly in WHERE clause ) can impact performance.  But it is not the case because in most case it is "inline" function or even "source" DB2 function which does not involve any additional function calling.
Conclusion
In most cases it is not a problem to migrate SQL Server build-in scalar function to DB2. Adding some consideration also migrating and keeping backward compatibility is possible.

niedziela, 1 lutego 2015

New version of JavaHotel, taxation

Introduction
I uploaded new version of JavaHotel application. Demo version (U/P user/user) is available here, source code here.
Taxation
While issuing a document (receipt, bill) a taxation is calculated using tax level assigned to the service.

Detailed information related to the document is also available. This information if stored in the database (as XML file) for every document issued.
Taxation information is also exposed in "Stay summary" window providing information about the current status of the stay.
Next step 
Receptionist journal registering the information about user activity.

Ubuntu 14.04 and USB tethering

Introduction
For some reason I was unable to connect to a hotel wi-fi network from my Ubuntu 14.04 desktop. What is more interesting, I connected without any problem my Kindle reader and mobile (Galaxy S2, Android 2.3.6). The hotel receptionist was very kind but also threw up his hand.
Solution
After spending some time googling on my mobile I found the solution and it was extremely simple, connect to the Internet through a mobile using USB tethering.

  1. Find a public place with wi-fi connection and install usbip package (yum install usbip)
  2. Load vhci-hcd module (modprobe vhci-hcd)
  3. Be sure that mobile is connected to wifi network
  4. Connect mobile and desktop via USB cable
  5. Enable USB Tethering in your mobile (Application -> Setting -> Wireless and network -> Tethering and portable hotsp -> Enable Tethering
And that's final, I have raised from death and connected to the world again.


wtorek, 30 grudnia 2014

DB2, federation and three-part names

Federation
Federation (comprehensive redbook on that topic can be downloaded here) allows correlating data from different data sources. Although InfoSphere Federation Server requires additional licensing, so called "homogenous" federation (comprising only DB2  databases, local and remote)  is available also in free DB2 Express Edition.
DB2 supports also three-part name referring to remote object. The dbname.schema.tablename notation allows migration from other databases (like Oracle or MS SQL). Although it works nicely unfortunately I found a problem which made me several nervous evenings.
Recreate the problem
Firstly enable DB2 instance for federation
db2 update dbm cfg using FEDERATED yes

db2stop
db2start
Create two databases
db2 create database test
db2 create database feder
Database 'test' will contain our tables and database 'feder' will act as a federated database.
Create two simple tables in 'test'
db2 "CREATE TABLE X1 (A1 INT, A2 INT)"
db2 "CREATE TABLE X2 (B INT)"
Then connect to 'feder' and create federation server
db2 "create wrapper drda"
db2 "create server testserver type db2/udb  version '10.5'  wrapper drda  authid user_name  password \"user_password\"  options(  add dbname 'TEST')"
db2 "CREATE USER MAPPING FOR PUBLIC SERVER testserver  OPTIONS (REMOTE_AUTHID 'user_name', REMOTE_PASSWORD 'user_password')"
Important: pay attention to lack of ' or " in user_name in CREATE SERVER \" in user_password and ' in CREATE USER MAPPING. You can spend several hours trying to discover it on your own !
Create a view (inner join)
db2 "create or replace view XXX as select * from testserver.db2inst1.x1,testserver.db2inst1.x2"
So far so good. But while testing the view there is a surprise.
db2 "select * from xxx"
SQL0158N  Liczba kolumn podana dla tabeli "DB2INST1.XXX" różni się od liczby
kolumn w tabeli wynikowej.  SQLSTATE=42811
There is no way to overcome it. Running this join directly (without passing through view) is successful. It seems that : view on join between remote tables accessed through three-part name is not working.
Solution 
Just come back to old DB2 school and use nicknames.
db2 create nickname testserver_db2inst1_x1 for testserver.db2inst1.x1
create nickname testserver_db2inst1_x2 for testserver.db2inst1.x2
db2 "create or replace view XXX as select * from testserver_db2inst1_x1,testserver_db2inst1_x2"

db2 "select * from xxx"

A1          A2          B          
----------- ----------- -----------

  Wybrano rekordów: 0.

Conclusion 
Hoping to be fixed in next release. Besides, three-part naming convention is very useful.

poniedziałek, 29 grudnia 2014

MS SQL -> DB2 migration, foreign keys

Introduction
During migration from MS SQL Server to DB2  I was facing a problem to generate foreign keys (in DB2 supported format) from MS SQL generated object script without having an access to MSSQL database. It is not a problem to recreate them manually if you have several objects but it could be a problem if you are dealing with hundreds or thousands of them.
So I decided to spend some time on creating a simple program (in Python) to accomplish the task automatically.
The source code is available here (Eclipse PyDev project).
Packages
The solution comprises several packages.
readfiles : read lines from several files (list of files in constructor). It flattens several files to one single reader.
atomizer : transforms input into sequence of atoms. For instance:

CREATE TABLE [dbo].[departments](
 [dept_no] [char](4) NOT NULL,
 [dept_name] [varchar](40) NOT NULL,
PRIMARY KEY CLUSTERED 
Atomizer will output: CREATE TABLE [dbo.department] ( [dept_no ... etc. It simply breaks input  (list of text lines) into sequence of elements ignoring spaces, line breaks etc.
tokenizer : transforms list of atoms into the sequence of recognized keywords ignoring elements out of importance here.
For instance.
Assuming set of constants describing keywords important here:
ALTER=0
SEMICOLON=1
TABLE=2
ADD=3
CONSTRAINT=4
FOREIGN=5
KEY = 6
REFERENCES = 7
BEGCOMMENT=8
ENDCOMMENT=9
CREATE=10
VIEW=11
FUNCTION=12
PROCEDURE=13
GO=14
The output will be the sequence: CREATE (constant 10) TABLE [dbo.department] (as single tekst) ( [dept_no ... (etc). Tokenizer makes further analysis more easier.
foreign : selects foreign key definition and prepares data structure: base table, constraint name, list of columns, reference table name and reference column list. Example:
ALTER TABLE [dbo].[dept_emp]  WITH CHECK ADD FOREIGN KEY([dept_no])
REFERENCES [dbo].[departments] ([dept_no])
ON DELETE CASCADE
or
ALTER TABLE [dbo].[dept_emp]  WITH CHECK ADD CONSTRAINT DEPT_NO_DEPARTMENT_FK FOREIGN KEY([dept_no])
REFERENCES [dbo].[departments] ([dept_no])
ON DELETE CASCADE
publish : takes data structure (describing foreign key definition as describe above) and prepares DDL in DB2 format. For instance:

ALTER TABLE dbo.dept_emp ADD CONSTRAINT "FK_dept_emp_departments" FOREIGN KEY
   (dept_no)
   REFERENCES dbo.departments
   (dept_no)
@
The main program

def test4():    
    R = readfiles.ReadFiles(INF)
    A = atomizer.Atomizer(R)
    T = tokenizer.TOKENIZER(A)
    F = foreign.ForeignSearcher(T)
    a = F.nextForeign()
    f = open("output/foreign_keys.db2","w")
    while a != None :
        s =  foreignDB2.foreignDB2(a)
        print s
        print ""
        print ""
        f.write(s)
        f.write("\n")
        f.write("\n")

        a = F.nextForeign()
    f.close()
The first four statements (objects construction) can be fused into a single statement.
Example input and output.
Conclusion
I found this approach useful. I also reused it in resolving several other problems.

  • Migrate only a subset foreign keys definition. Read firstly list of tables (without foreign keys) already migrated and emits only foreign keys related to them.
  • Prepare list of all objects (tables, view, UDF and SP) in MS SQL object script. 

niedziela, 14 grudnia 2014

Byliśmy na koncercie

22 listopada 2014 roku byliśmy na przedstawieniu opery Richarda Straussa "Ariadna na Naksos" w Filharmonii Narodowej, występ podobał się bardzo, chociaż nie wszystkim byliśmy zachwyceni.
"Ariadna na Naksos" to jedna z tych oper, za którą tak bardzo lubimy Richarda Straussa. W Polsce rzadko wystawiana, na scenie Filharmonii po raz pierwszy. Przyczyną są najprawdopodobniej trudności realizacyjne, wymaga zaangażowania aż 17 solistów i niezbyt licznej, ale za to lubiącej wyzwania orkiestry, zdolnej sprostać skomplikowanej partyturze. Na szczęście w Warszawie w zupełności się to udało, i to nawet w całości krajowymi siłami z pomocą tylko czterech śpiewaków z importu. Szczególnie się podobała Anna Simińska w roli żywiołowej Zerbinetty, po brawurowym wykonaniu niezwykle efektownej arii "Großmächtige Prinzessin" artystka zebrała długie i zasłużone oklaski. Znakomicie także śpiewała Meagan Miller jako Primadonna. Potężny głos tenora Andreasa Schagera zdawał się rozsadzać salę Filharmonii, jakby artysta zapomniał, że nie śpiewa roli boga wojny Marsa czy władcy piorunów Jowisza, a boga winorośli Bachusa.
Wadą była niestety sama forma przedstawienia, gdyż było to przedstawienie koncertowe, bez akcji scenicznej. Brak scenicznego ruchu nadrabiał ekran na którym był wyświetlany polski przekład. Ale pomimo tego prolog, którego znaczną część wypełniają monologi i recytatywy zwyczajnie nużył. W drugim akcie znacznie lepiej było nawet zrezygnować z śledzenia napisów, a po prostu wsłuchać się w muzykę, która jest wspaniała i niezwykła.
Akcja opery jest statyczna, są tutaj dwa akty i na scenie nie musi się wiele dziać. Jednak zgubił się sam zamysł opery, która jest przecież zestawieniem kontrastów. Przeszłości i teraźniejszości (prolog i opera), świata rzeczywistego i mitycznego, wzniosłości i trywialności (Kompozytor i Baletmistrz), tragedii i komedii. Sztuki przez duże S, ale pozbawionej pieniędzy kontra duże pieniądze szukające sztuki przez małe s czy nawet śmiesznych sztuczek.
Muzyka Straussa, zwłaszcza w tak znakomitym wykonaniu, na szczęście zawsze brzmi i zachwyca tak samo, jednak brak scenicznej inscenizacji wyraźnie ubożył odbiór tego dzieła. Trzeba mieć nadzieję, że to wystawienie nie zakończy kariery "Ariadny na Naksos" na warszawskich scenach i będziemy mieć okazję poznać dzieło Straussa w kompletnej formie.