Photo by Markus Spiske on Unsplash

Spark from_avro() and to_avro() usage

In Spark, avro-module is an external module and needed to add this module when processing Avro file and this avro-module provides function to_avro() to encode DataFrame column value to Avro binary format,…

0 Comments

March 21, 2020

Spark Streaming – Kafka messages in Avro format

This article describes Spark Structured Streaming from Kafka in Avro file format and usage of from_avro() and to_avro() SQL functions using the Scala programming language. Spark Streaming Kafka messages in…

7 Comments

March 23, 2019