Blog do projektu Open Source JavaHotel

niedziela, 29 kwietnia 2018

KVM, HortonWorks and IBM Spectrum Scale

IBM Spectrum Scale (former GPFS) is a clustered file system giving single access point to distributed parallel nodes. HDP is Hadoop implementation created and supported by HortonWorks. Both work together smoothly, GPFS can replace HDFS in a transparent way.
If you are entitled to use IBM Spectrum Scale, it is tempting to install IBM Spectrum Scale locally and explore its capabilities without setting up huge infrastructure.
Unfortunately, although there is plenty of ample information available, it is not easy to make the first step and just install it.
So I decided to fill the gap and install HDP HortonWorks on the top of IBM Spectrum Scale (GPFS) using KVM virtualization. I'm not paying too much attention to stuff like tunning, mirroring, replication, configuration etc. Just install using a default setting as much as possible to give the first taste of this fantastic feature.
More details can be found here.
Before you start:
  • Make sure you are entitled to use IBM Spectrum Scale that way, review license agreement. IBM Spectrum Scale is not publicly available.
  • Do not start unless you at least 32GB memory, otherwise, 4 KVM with at least 6 GB memory will kill your machine.

Brak komentarzy:

Prześlij komentarz