L̶u̵m̶i̵n̷o̴u̶s̶m̶e̵n̵B̶l̵o̵g̵

2021-05-11 19:19:00 Learn concepts not frameworks

In the world of engineering and data analytics, enthusiastic people often have strong opinions about using a particular platform. We've all come across such a person — someone who zealously promotes Apache Spark or pushes for all data work to be done using HQL. There's a strong emphasis on the tools and not so much on the problem-solution pair.

Sometimes this behavior is driven by a desire for standardization, aligning team skills, and simplification of the hiring process. Those are certainly valid considerations. But more often than not, I've seen people zealously defend a tool simply because they are passionate about it or, even worse, have built their professional careers around it.

I wish concepts were emphasized a bit more. In the real world, knowing Java doesn't make you a programmer.

Knowing C++ syntax doesn't make you a programmer.

Knowing concepts is what makes you a great programmer. Problem solving skills make you a great programmer.

Simply put, you should never build your professional career around a tool. Tools and applications come and go, and what's hot today may go bad tomorrow. That alone should give you a reason to pause before advocating for technology too quickly. Ask anyone who has worked with JavaScript.

The Silver Bullet Syndrome - Hadi Hariri

#whining Open / Comment

2021-05-10 19:19:00 To build this feature store, DoorDash benchmarked several key-value stores—Redis, Cassandra, CockroachDB, ScyllaDB, and YugabyteDB—before sticking with Redis. More details on their benchmarking process and optimizing Redis in their excellent write-up.

#ml Open / Comment

2021-05-09 16:16:00 Kafka Visualization

https://softwaremill.com/kafka-visualisation/

#big_data Open / Comment

2021-05-05 16:16:00 Apache Software Foundation (ASF) has announced the retirement to its "Attic" of at least 19 open source projects, 13 of which are big data-related and ten of which are part of the Hadoop ecosystem.

https://www.zdnet.com/article/apache-software-foundation-retires-slew-of-hadoop-related-projects/

I've really never used any of these projects, but could this be a signal that the Hadoop era is coming to an end?

It's another proof that one shouldn't build one's career around a particular tool, but rely on concepts and understanding of the industry. Technology is a very volatile thing, especially in fast-paced industries like Big Data and Data Science. Especially when industry consolidation is still in process.

#big_data Open / Comment

2021-04-30 16:16:00 Recent Advances in Language Model Fine-tuning

By Sebastian Ruder:

https://ruder.io/recent-advances-lm-fine-tuning/ Open / Comment

2021-04-28 16:16:00 Google has made its AutoML algorithm public

https://github.com/google/model_search

#ml Open / Comment

2021-04-13 16:16:00 Let's talk about a couple of core concepts in data space and how they relate to each other and how they started.

ACID vs BASE: Comparison of two Design Philosophies

CAP and PACELC theorems in plain English Open / Comment

2021-04-12 16:16:00 Open / Comment

2021-04-11 16:16:00 Remember I told you about numbers from -5 to 255 interned. That is, the numbers are preliminary placed and cached in memory. So, there is more to it.

You can access this memory if you want and change value. Say, change literal 4 to value 5. I'll probably be damned by many people for this post, but here is a sample code:

>>> import ctypes
>>> ctypes.memmove(id(4) + 24, id(5) + 24, 8)
>>> print(2 * 2) # 5

Have fun debugging!

P.S. Try to replace 0 with 1

#python Open / Comment

2021-04-10 16:16:00 There are compilers for Python code. And not only JIT ones like Numba, but also ordinary ones. For example, Cython, Nuitka that compiles your Python code into true machine instructions rather than interpreted.

Why?

Basically for the sake of both performance gains and a more portable runtime. So basically it's for the guys who shouldn't use Python at the first place.

#python Open / Comment