dgadiraju
11/4/2017 - 3:12 PM

write-processed-data-to-hdfs-compressed-gzip.scala

write-processed-data-to-hdfs-compressed-gzip.scala
content_copyfile_download
sqlContext.setConf("spark.sql.parquet.compression.codec","gzip");
dataFrameResult.write.parquet("/user/cloudera/problem1/result4a-gzip");
sqlResult.write.parquet("/user/cloudera/problem1/result4b-gzip");
comByKeyResult.write.parquet("/user/cloudera/problem1/result4c-gzip");