Scala: how to run a shell command from the code level?

To execute the shell command, you must import the scala.sys.process library and then use the available Domain Specific Language (DSL) defined with an exclamation mark (!).

# Import library
import scala.sys.process._

# Run shell command to create directory in HDFS
val targetPath = "/bigdataetl/data"
s"hdfs dfs -mkdir -p ${targetPath}" !


Apache Spark

Using Apache Spark and writing in Scala, you may find that you will also need to perform some operation on files or directories. You can also use the above library as much as possible.

# Delete file from HDFS
s"hdfs dfs -rm /bigdata_etl/temp_file.dat" !


If you enjoyed this post please leave the comment below or share this post on your Facebook, Twitter, LinkedIn or another social media webpage.
Thanks in advanced!

Please follow and like us:

Leave a Reply

Close Menu
Social media & sharing icons powered by UltimatelySocial

Enjoy this blog? Please spread the word :)