Run Cloudera QuickStart using Docker
In this short post I will show how you can run the Cloudera QuickStart using Docker. As you know from my previous post I am big fan of dockers and…
Continue Reading Run Cloudera QuickStart using Docker
Photo by Annie Spratt on Unsplash
How to save data from Hive (Parquet, Text, ORC) to CSV file or any different file type?
In this post I will show you few ways how you can export data from Hive to csv file. For this tutorial I have prepared hive table "test_csv_data" with few…
Continue Reading How to save data from Hive (Parquet, Text, ORC) to CSV file or any different file type?
Talend: Kafka and MongoDB – real-time streaming
In today's world, we often meet requirements for real-time data processing. There are quite a few tools on the market that allow us to achieve this. At the forefront we…
Continue Reading Talend: Kafka and MongoDB – real-time streaming
Apache Spark: how to rename or delete a file from HDFS?
In this short post I will show you how you can change the name of the file / files created by Apache Spark to HDFS or simply rename or delete any file.
Continue Reading Apache Spark: how to rename or delete a file from HDFS?
Scala: How to run a shell command from the code level?
In this post I will show you how to run the shell command by programming in Scala and how you can use it in Apache Spark.
Continue Reading Scala: How to run a shell command from the code level?
Apache Spark: How to save DataFrame as a single file on HDFS?
If you want to save DataFrame as a file on HDFS, there may be a problem that it will be saved as many files. This is the most correct behavior and it results from the parallel work in Apache Spark.
Continue Reading Apache Spark: How to save DataFrame as a single file on HDFS?
Apache Spark: How to check if the file exists on HDFS?
We will use the FileSystem and Path classes from the org.apache.hadoop.fs library to achieve it.
Continue Reading Apache Spark: How to check if the file exists on HDFS?
Connection configuration between Talend and Cloudera
Cloudera is one of the three major players in the market alongside Hortonworks and MapR, which distributes the Hadoop general-interest.
Continue Reading Connection configuration between Talend and Cloudera
Apache Spark: Machine Learning – predicting diabetes in patients
Today I will show you how you can use Machine Learning libraries (ML), which are available in Spark as a library under the name Spark MLib.
Continue Reading Apache Spark: Machine Learning – predicting diabetes in patients
How to install Apache Spark Standalone in CentOs?
In this tutorial I will show you how you can easily install Apache Spark in CentOs
Continue Reading How to install Apache Spark Standalone in CentOs?
