Scala: How to run a shell command from the code level?

To execute the shell command, you must import the scala.sys.process library and then use the available Domain Specific Language (DSL) defined with an exclamation mark (!).

# Import library
import scala.sys.process._

# Run shell command to create directory in HDFS
val targetPath = "/bigdataetl/data"
s"hdfs dfs -mkdir -p ${targetPath}" !

Apache Spark

Using Apache Spark and writing in Scala, you may find that you will also need to perform some operation on files or directories. You can also use the above library as much as possible.

# Delete file from HDFS
s"hdfs dfs -rm /bigdata_etl/temp_file.dat" !

If you enjoyed this post please add the comment below or share this post on your Facebook, Twitter, LinkedIn or another social media webpage.
Thanks in advanced!

0 0 vote
Article Rating
Notify of
Inline Feedbacks
View all comments