Visualize car data with Brunel and Scala

There is a sample in IBM Data Science Experience "Visualize car data with Brunel". But this sample notebook is written in Python, PySpark. So I transformed it to Scala just receiving the same result but using Scala syntax. I added some comments to explain the code.
The result is published here.
To run it:

• Download Cars+.ipynb notebook
• Upload to Jupyter with Apache Toree-Scala (Spark) kernel enabled
• Enjoy

Next version of Civilization The Board Game

Introduction
I deployed next version of my computer implementation of  Civilization The Board Game. The implementation consists of three parts:

• Civilization Engine 
• Civilization Web Interface
• Demo. Demo version is deployed to Heroku, free quota. Please wait several minutes unless the container is brought back to life.

New features
I implemented new, more user-friendly interface.
Single user, training game.

Two players game, real melee battle.

The opponent deck is inactive, informative only.

More user-friendly features
If figure (scout or army) is in the corner between two hidden tiles, a dialog to select one to be revealed comes up.

Figures can be stacked. If a player decided to move stacked figures, a dialog to select figures to move pops up.

After selecting an action in the left panel, squares, where action could be conducted, are highlighted.

Next steps
Implement

• Spend trade to rush the production
• Send production from scout to city
• Buying units
• Battles (?)

New version of Civilization the Board Game

Introduction
I deployed next version of my computer implementation of  Civilization The Board Game. The implementation consists of three parts:

• Civilization Engine 
• Civilization Web Interface
• Demo. Demo version is deployed to Heroku, free quota. Please wait several minutes unless the container is brought back to life.

New features

• Game progress saved
• Game resume
• Two players game

Game progress saved

The game is saved constantly in redis key/value database. It is done automatically in the background in a transparent way, there is no "Save" button. The server side of the game is stateless, it is only computational engine, redis datastore is a memory cache. Redis is accessed through an interface, I'm going to prepare HBase version as a warmup. Stateless is a very important feature, in the imaginative future when thousands of players are swarming, load balancing and traffic redirection can be applied. 

Game resume

A game can be left and resumed at any time. Just select a game and the player is pushed immediately into the middle of the battle.

Also, two players game can be resumed but second players should join to continue the game.
Two players game
Two options of creating new game are available: "Training" and "Two players game".

Training is a single player game, just to get one's hand in. "Two players game" is more serious matter, test yourself in hand-to-hand battle. Select opponent civilization you want to fight against and wait for the contester.

Joining the game
Select the game from the waiting list and the contester together with one who threw down the gauntlet are moved into the middle of the duel.

Leaving the game
The player can leave the game by clicking the close icon in the upper right corner of the board.

Leaving the game legally unlocks the game and remove it from the waiting list. But because the game is saved automatically while playing, event after abruptly closing the browser the game can be resumed exactly at the same stage whatsoever. The game is removed from waiting list after 24 hours of inactivity.
Next steps
Make map and player deck more user-friendly.

DB2, UTL_ENCODE, BASE64_DECODE, BASE64_ENCODE

I created an implementation of the two methods from Oracle UTL_ENCODE package.  It is implemented as UDF Java function and DB2 module. In Java, it is simply a utilization of JVM Base64 package. More time consuming was preparing DB2 signature and test according to this article.
Full source code is here.

Civilization The Board Game

Introduction
For some time, I became a fan of Civilization The Board Game. I found it more engaging, dynamic and enthralling than the computer game. It is like comparing real melee combat with the bureaucratic war waged behind the office desk.
And an idea stirred me up to move the game to the computer screen. Avoid the stuff piling up on the table, train and test ideas without spreading the board game and allow players in remote locations to fight.
For the time being, my idea ended up in two projects.
CivilizationEngine here
Civilization UI here
Demo version on Heroku: https://civilizationboardgame.herokuapp.com/  (wait a moment until dyno is activated, it is a free quota).
Every project comes with its own build.xml file allowing creation of target artifact.
General design principles
The solution consists of two separate projects: Civilization Engine and Civilization UI. I decided that all game logic and state is managed by back end engine. The UI, as the name suggests, is focused only on displaying the board game and allowing the user to execute a command. The command is sent to the server, server changes the game state and UI is receiving the current game state and updates the screen.
The game is nothing more like moving from one game state to another. Every change is triggered by the command. At every moment, it is possible to restore the current game state by setting the initial board and replaying all commands up to the point.
Data is transmitted between engine and UI in JSON format.
Civilization Engine
Civilization Engine is created as IntelliJ IDEA Scala project, can be imported directly from GitHub.
Why Scala? I found it very appropriate here. Most of the operations are related to list walking through, list looking up, filtering, mapping and Scala is an excellent tool for that. If I decided to use Java probably the code would pump up twice even with Java8 streaming features.
I'm very fond of this command (full source) :

  def itemizeForSetSity(b: GameBoard, civ: Civilization.T): Seq[P] =
    getFigures(b, civ).filter(_.s.figures.numberofScouts > 0).map(_.p).filter(p => SetCityAction.verifySetCity(b, civ, p, Command.SETCITY).isEmpty)

It yields all points where a new city can be set.

• Find all figures on the board belonging to a civilization
• Single out squares with at least one scout
• Map squares to points
• Verify if the point is eligible for city setting using SetCityAction.verifySetCity

All stuff in a single line.

A general outline of the project

• resources, game objects (JSON format) used in the game: tiles, squares, objects (now TECHNOLOGIES only)
• gameboard , class definitions
• objects , enumerations and classes related to game artifacts
• helper, game logic, I found more convenient to put them as helper, object class then as methods in Gameboard class.
• io, methods regarding reading and writing data in JSON format. I'm using a dependency PlayJSON package.
• I, external interface

Brief interface description

• getData(LISTOFCIV), list of civilizations available
• getData(REGISTEOWNER), generates new game and returns unique token to be used in further communication
• getData(GETBOARDGAME), returns current game state
• executeCommand,  executes next command
• itemizeCommand, provides all possible parameters for a particular command. For instance: for StartOfMove it brings all points where figure movement are allowed to commence.

So far, only a few commands are implemented

• SetCapital
• SetArmy
• SetScout
• EnfOfPhase
• BuyScout
• BytArmy
• MoveFigure
• RevealTile
• SetCity

User Interface

For the time being, so ugly that only a mother or father could love it. More details: look here.

Next steps

Implementation of game persistence. Because of Heroku limitation, I cannot use disk file system as a mean. I'm planning to use Redis, there is a free quota for this service in Heroku. Redis will be used to store the games and also as a cache. This way, the server part will be completely stateless. Every step will consist of restoring the game from Redis, executing a command and storing the updated game to Redis again.

Hosts, simple bash tool to run commands on several hosts

BigInsights  (IBM Hadoop) requires a number of prerequisites to have the cluster consistent. For instance: /etc/hosts file should be the same on all hosts. There are plenty of tools available, but finally, I decided to create a small tool on my own.
The tool is available here (branch hosts).
There are several simple bash procedures allowing copying and executing a single command on all hosts in the cluster. For instance: install a required package using yum command on all hosts.
Basically, two main tasks are implemented:

• Share file across hosts (for instance /etc/hosts)
• Run a single command on all hosts (for instance yum install package)

These two simple tools fulfill almost all tasks necessary to prepare and run the multi-host installation of BigInsights and IBM streams.

More details and description here.

Dockerize IBM Streams

It is very convenient to run IBM Streams in Docker container to avoid huge VM overhead. There is one project available, but my plan is not so ambitious.
The solution is described here, Dockerfile file is also available there. It is not full automation, it is rather several pieces of advice how to set up Docker container with running IBM Streams domain and instance inside, just lightweight Virtual Machine easy to set up and calm down.
But it comes with one serious limitation. Only single host standalone installation is possible. Multihost installation requires resolving IP-DNS mapping and I failed to overcome this problem.
But standalone installation is enough for developing, testing and evaluation. I will keep going to support multihost also.