Databricks

Greek Architectures of Data Processing

Three popular Data processing architecure for big and small data on cloud. These cover various scenarios for both batch, realtime, small and big data. The links take to the dedicated blog for each architecture

How to structure the Data Lake

The key reasons for the need of good data lake structure are: 1) Security: need of role-based security on the lake for read access. 2) Extendibility: it should be easy to extend the lake after first round and more systems can be added 3) Usability: it should be easy to use and find the data in the lake and the users should not get lost 4) Governance: it should be simple to apply governance practices to the lake in terms of quality, metadata management and ILM

Lambda Architecture using Databricks

From technology point of view Databricks is becoming the new normal in data processing technologies, in both Azure and AWS. This post provides a view of lambda architecture and uses Databricks at front and center. Databricks has capabilities to replace multiple tools and those are described in bit detail below

5 Things to get best out of Azure Databricks

Databricks has become the new normal in the data processing in cloud. If you are using or plan to use Azure Databricks, this post is will guide you on some interesting things that you can plan to investigate as you start.