My Tech Journal

The problem with taking over an existing project in between is there are always blindspots you aren’t aware of. This week a cassandra issue was reported on customer site which stated that a particular cassandra query was unable to execute successfully and was retrying indefinitely. When we started investigating we...

For a java developer scala is pretty much familiar - you can code without going deep into the details. Which can do the job, but magic happens when you explore it and try to write code the scala way. Once such thing is scala’s flexible import. We all know that...

Every time I wanted to use scala collection, a question would popup - whether to use immutable collection as var or use mutable collection. As per scala collection performance they seem pretty identical, but still the question is what exactly is difference, however minor it may be. First thing I...

I faced this issue when working on a project last week. So I had to add a Kafka consumer in the project in order to write an integration test case. Now kafka consumer is pretty straightforward when using akka-stream, but the project had an earlier version of Akka (2.4.8) on...

This is an all in one tutorial for creating an analytics dashboard using Django, which is a python based web-framework. We have used Bootstrap (CSS framework) for frontend, HighCharts (Javascript charting library) for charts, and Django as web-framework Create Skeleton Website #1. Install Django pip3 install django #2. Create project...

Spark Release 2.4.0 Apache Spark 2.4.0 is the spark’s latest version - the fifth release in 2.x line - released in Nov, 2018. This release has some major features such as: Barrier Execution Mode for better integration with deep learning frameworks, 30+ built-in and higher-order functions to deal with complex...

Why Do You Need Virtal Environment? Imagine you have an application that needs version 2.7 of Python, but another application requires version 3.0. How can you use both these applications? Whhat if you want to install an application and leave it be? If an application works, any change in its...

This article in our Spark Tutorial series demonstrates the reading of data into Spark DataFrame and applying different transformations on it. Prerequisites: Set up Spark development environment and review the Spark Fundamentals. Objective: To understand Spark DataFrames and load data into Apache Spark. Overview It’s much easier to program in...