Skip to content
Big Data & ETL
  • Big Data
  • Spark
  • Airflow
  • Kafka
  • ETL
  • Talend
  • RSS
  • English
  • Polski
Menu Close
  • Big Data
  • Spark
  • Airflow
  • Kafka
  • ETL
  • Talend
  • RSS
  • English
  • Polski

Big Data

Read more about the article How to Apache Spark Break DAG lineage – Do you know these 3 cool methods?
Photo by Ayla Verschueren on Unsplash

How to Apache Spark Break DAG lineage – Do you know these 3 cool methods?

  • Post category:Apache Spark/Articles/Big Data
  • Reading time:10 mins read

In this post, I will introduce you to 3 methods how to Apache Spark Break DAG lineage. It's very possible…

Continue ReadingHow to Apache Spark Break DAG lineage – Do you know these 3 cool methods?
Read more about the article How to copy files from one directory to another on HDFS (Hadoop)? – Start doing it the 1 right way!
Hadoop Logo

How to copy files from one directory to another on HDFS (Hadoop)? – Start doing it the 1 right way!

  • Post category:Articles/Big Data/Cloudera
  • Reading time:7 mins read

We often encounter the need to copy data between directories on HDFS on Hadoop. [ How to copy files from…

Continue ReadingHow to copy files from one directory to another on HDFS (Hadoop)? – Start doing it the 1 right way!
Read more about the article Apache Kafka How to delete data from Kafka topic? – you probably didn’t know these 2 cool methods!
Photo by John Schnobrich on Unsplash

Apache Kafka How to delete data from Kafka topic? – you probably didn’t know these 2 cool methods!

  • Post category:Apache Kafka/Articles/Big Data
  • Reading time:5 mins read

When working with Apache Kafka, there may be a situation when [ Apache Kafka How to delete data from Kafka…

Continue ReadingApache Kafka How to delete data from Kafka topic? – you probably didn’t know these 2 cool methods!
Read more about the article Apache Spark ReduceByKey vs GroupByKey – differences and comparison – 1 Secret to Becoming a Master of RDD!

Apache Spark ReduceByKey vs GroupByKey – differences and comparison – 1 Secret to Becoming a Master of RDD!

  • Post category:Apache Spark/Articles
  • Reading time:5 mins read

In this post I will try to introduce you to the main differences between Apache Spark ReduceByKey vs GroupByKey methods…

Continue ReadingApache Spark ReduceByKey vs GroupByKey – differences and comparison – 1 Secret to Becoming a Master of RDD!
Read more about the article Run Cloudera QuickStart using Docker – easy steps & setup in 5 mins!

Run Cloudera QuickStart using Docker – easy steps & setup in 5 mins!

  • Post category:Articles/Big Data/Cloudera/Docker
  • Reading time:9 mins read

In this short post I will show how you can run the Cloudera QuickStart using Docker. As you know from…

Continue ReadingRun Cloudera QuickStart using Docker – easy steps & setup in 5 mins!
Read more about the article How to save data ORC Parquet Text CSV in Hive file or any different file type? – 4 types 1 easy approach!
Photo by Annie Spratt on Unsplash

How to save data ORC Parquet Text CSV in Hive file or any different file type? – 4 types 1 easy approach!

  • Post category:Apache Hive/Articles/Big Data/Cloudera
  • Reading time:7 mins read

In this post I will show you how to save data ORC Parquet Text CSV in Hive in few ways…

Continue ReadingHow to save data ORC Parquet Text CSV in Hive file or any different file type? – 4 types 1 easy approach!
Read more about the article Talend Kafka MongoDB Docker-Compose Real-Time Streaming – Integrate Them Together in 3 Mins?

Talend Kafka MongoDB Docker-Compose Real-Time Streaming – Integrate Them Together in 3 Mins?

  • Post category:Apache Kafka/Articles/Big Data/Docker/Docker-Compose/ETL/MongoDB/Talend/Ubuntu
  • Reading time:20 mins read

In today's world, we often meet requirements for real-time data processing (Talend Kafka MongoDB Docker-Compose real-time). There are quite a…

Continue ReadingTalend Kafka MongoDB Docker-Compose Real-Time Streaming – Integrate Them Together in 3 Mins?
Read more about the article Apache Spark Rename Or Delete a File HDFS – Great Example In 1 Minute?

Apache Spark Rename Or Delete a File HDFS – Great Example In 1 Minute?

  • Post category:Apache Spark/Articles/Big Data/Programming Languages/Tips
  • Reading time:6 mins read

In this short post I will show you how you can using Apache Spark rename or delete a file HDFS.…

Continue ReadingApache Spark Rename Or Delete a File HDFS – Great Example In 1 Minute?
Read more about the article How to run shell command in Scala from the code level – check great code snippet in 1 minute?

How to run shell command in Scala from the code level – check great code snippet in 1 minute?

  • Post category:Apache Spark/Articles/Big Data/Programming Languages/Tips
  • Reading time:2 mins read

How to run shell command in Scala [ How to run shell command in Scala ] To execute the shell…

Continue ReadingHow to run shell command in Scala from the code level – check great code snippet in 1 minute?
Read more about the article Apache Spark Save DataFrame As a Single File HDFS – 1 Min Solution?

Apache Spark Save DataFrame As a Single File HDFS – 1 Min Solution?

  • Post category:Apache Spark/Articles/Big Data
  • Reading time:5 mins read

Problem In this tutorial I will show the example when using Apache Spark Save DataFrame as a single file HDFS.…

Continue ReadingApache Spark Save DataFrame As a Single File HDFS – 1 Min Solution?
  • 1
  • 2
  • 3
  • Go to the next page

Android (5) Apache Airflow (6) Apache Hive (3) Apache Kafka (2) Apache Spark (14) Big Data (21) Cloudera (4) Docker (13) Docker-Compose (9) ETL (5) Excel (3) IntelliJ (3) Java (6) Maven (7) Microsoft Azure (2) MySQL (4) Oracle (4) Scala (3) Spring Boot (3) SQL Developer (5) SQL Server (6) Talend (7) Teradata (13) Tips (37) Ubuntu (10) Windows (4)

Copyright 2022 - by BigData-ETL
Icon made by Freepik from www.flaticon.com
GDPR
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
Do not sell my personal information.
Cookie SettingsAccept
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT