In Spark SQL, in order to convert/cast String Type to Integer Type (int), you can use cast() function of Column class, use this function with withColumn(), select(), selectExpr() and SQL…
In PySpark SQL, using the cast() function you can convert the DataFrame column from String Type to Double Type or Float Type. This function takes the argument string representing the…
Spark SQL select() and selectExpr() are used to select the columns from DataFrame and Dataset, In this article, I will explain select() vs selectExpr() differences with examples. Both these are…
In PySpark, the substring() function is used to extract the substring from a DataFrame string column by providing the position and length of the string you wanted to extract. In…
In PySpark, you can cast or change the DataFrame column data type using cast() function of Column class, in this article, I will be using withColumn(), selectExpr(), and SQL expression…
To change the Spark SQL DataFrame column type from one data type to another data type you should use cast() function of Column class, you can use this on withColumn(),…
Spark DataFrame example of how to add a day, month and year to a Date column using Scala language and Spark SQL Date and Time functions.