scala
Recent Posts
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy
To find out more, including how to control cookies, see here: Cookie Policy
Categories
- Design (2)
- Elasticsearch (3)
- Golang (5)
- hadoop (2)
- Java (9)
- linux (1)
- Performance (1)
- Scala (2)
- Security (3)
- Spark (2)
- Spring (1)
- Spring Batch (1)
- Spring Boot (1)
- sqoop (1)
- UI (1)
- unix (1)
- Vim (2)
Tags
annotation
apache hive 3
cdp
cloudera
commands
CORS
dataframe
design pattern
DNS
elastic
elasticsearch
facets
go
golang
hadoop
hdfs
hive
http
ip
java
jms
jpa
JpaRepository
junit
locking
mq
mysql
nginx
oracle
proxy
queue
rdbms
rdd
Repository
resolution
reverseproxy
scala
spark
spring boot
springboot
spring data
sql
sqoop
string
vim
Recent Posts
Categories
- Design (2)
- Elasticsearch (3)
- Golang (5)
- hadoop (2)
- Java (9)
- linux (1)
- Performance (1)
- Scala (2)
- Security (3)
- Spark (2)
- Spring (1)
- Spring Batch (1)
- Spring Boot (1)
- sqoop (1)
- UI (1)
- unix (1)
- Vim (2)
Tags
annotation
apache hive 3
cdp
cloudera
commands
CORS
dataframe
design pattern
DNS
elastic
elasticsearch
facets
go
golang
hadoop
hdfs
hive
http
ip
java
jms
jpa
JpaRepository
junit
locking
mq
mysql
nginx
oracle
proxy
queue
rdbms
rdd
Repository
resolution
reverseproxy
scala
spark
spring boot
springboot
spring data
sql
sqoop
string
vim
Find Us
Address
123 Main Street
New York, NY 10001
Hours
Monday–Friday: 9:00AM–5:00PM
Saturday & Sunday: 11:00AM–3:00PM
%d bloggers like this:
Generate Sequential and Unique IDs in a Spark Dataframe
Apache Spark is an open source, general-purpose distributed computing engine used for processing and analyzing a large amount of data. Hence, adding sequential and unique IDs to a Spark Dataframe is not very straight forward, because of distributed nature of it.
Share this:
Like this:
Continue Reading
Spark Partitions with Coalesce and Repartition (hash, range, round robin)
One main advantage of the Apache Spark is, it splits data into multiple partitions and executes operations on all partitions of data in parallel which allows us to complete the job faster.While working with partition data we often need to increase or decrease the partitions based on data distribution. Methods repartition and coalesce helps us to repartition.
Share this:
Like this:
Continue Reading
Scala String Interpolation
Introduction String Interpolation refers to substitution of defined variables or expressions in a given String with respected values. String Interpolation allows users to embed variable references directly in processed string literals.. read more…
Share this:
Like this:
Continue Reading
Multiline Strings in Scala
You want to create multiline strings within your Scala source code, like you can with the heredoc syntax of other languages and help in escaping quotes and other symbols. A heredoc is a way. read more…
Share this:
Like this:
Continue Reading