Get Mystery Box with random crypto!

L̶u̵m̶i̵n̷o̴u̶s̶m̶e̵n̵B̶l̵o̵g̵

Logo of telegram channel iamluminousmen — L̶u̵m̶i̵n̷o̴u̶s̶m̶e̵n̵B̶l̵o̵g̵ L
Logo of telegram channel iamluminousmen — L̶u̵m̶i̵n̷o̴u̶s̶m̶e̵n̵B̶l̵o̵g̵
Channel address: @iamluminousmen
Categories: Technologies , Blogs
Language: English
Subscribers: 335
Description from channel

(ノ◕ヮ◕)ノ*:・゚✧ ✧゚・: *ヽ(◕ヮ◕ヽ)
helping robots conquer the earth and trying not to increase entropy using Python, Big Data, Machine Learning
http://luminousmen.com
License: CC BY-NC-ND 4.0

Ratings & Reviews

2.50

2 reviews

Reviews can be left only by registered users. All reviews are moderated by admins.

5 stars

0

4 stars

0

3 stars

1

2 stars

1

1 stars

0


The latest Messages 2

2021-07-18 16:16:00 «There's this one quiet guy who's the absolute beast of excel spreadsheets analytics.»

Cool introductory video about data engineering for those who are not in the field yet.





Btw, we have this one book to help new or not that new engineer, just FYI

#big_data
218 viewsedited  13:16
Open / Comment
2021-07-14 19:19:00 No one knows how to manage programmers — and everyone comes up with workarounds instead of solutions. One reality is the product, and the real engineers who improve it, because they want to and because they can.

And guess what — Jira board is bullshit. I've seen horrible products that had a Kanban board in perfect condition, and great products with several developers that told everyone to go fuck themselves. And the only person who can synchronize the real state of the project with the Jira tickets is a Team Lead.

We live in a horrible world where Jira ticket circulation is considered a positive thing, while the real improvement of the product remains unnoticed. I think I'll get another drink.

#soft_skills #leadership
220 views16:19
Open / Comment
2021-07-13 19:19:00 I'm sorry, but I think we need to sort out the terms in Data Engineering - I'm starting to get confused myself. Let me give you some thinking on what I usually mean:

Pipeline - a sequence of stages

Stage/Job - includes several tasks

Task - a single atomic process to be executed (script, command, utility)

Workflow - automation technology of a business process

The difference between pipeline and workflow:
Pipeline a clearly described process in which tasks are executed sequentially, or processes executed in parallel.
Workflow is usually non-linear and has an often abstract human description, processes may not run in parallel. Workflow has branches and loops.

#big_data
201 views16:19
Open / Comment
2021-07-13 10:10:00 While there are a lot of advantages of using Cloud Dataproc, there are a lot of pitfalls that inexperienced engineers can run into. And in this post, we'll take a look at some things you should take into account when you're building Cloud Dataproc-based pipelines or architecture using it.

https://luminousmen.com/post/things-to-consider-while-running-google-cloud-dataproc
176 views07:10
Open / Comment
2021-07-12 19:19:00 Who is responsible for the engineering team failures?

If you're a Team Leader of a team then it should be your moto: Failures are always mine, and successes are always teams.

I think this is a good principle to build a good team and good company.

Take care of the engineers. Customers, management, other departments often pour heaps of shit on them when the team makes mistakes. If you are a Team leader, you need to filter the negative and convert it to a positive. Take the hit for team mistakes. If the team failed, it means you build up bad processes. If the team succeed, it is their success and it doesn't matter if it is true or not.

When I first tried myself as a team lead I was angry at one of my collegues who made a mistake, and since I'm a fairly straightforward person — I said it directly.

Quite quickly I started to understand that it's a dead-end road and does not solve the problem. I began to understand that it is not engineers who work for a leader, but the leader exists to serve people. The goal of a leader is to serve. The goal of a team lead is to help the team.

Do not play a blame game, you need to look for the root cause of the problem. And most likely, it lies in the organization of work. Just discuss these problems quietly in retrospective meeting, find the root cause, create action items and do them.

#soft_skills #leadership
162 views16:19
Open / Comment
2021-07-10 16:16:00 Bad code can be written in any paradigm
171 views13:16
Open / Comment
2021-07-08 19:19:00 Good software architecture diagrams assist with communication inside/outside of software development/product teams, efficient onboarding of new staff, architecture reviews/evaluations, risk identification, threat modelling, etc.

The C4 model is an easy to learn, developer friendly approach to software architecture diagramming.

https://c4model.com/
215 views16:19
Open / Comment
2021-07-07 16:16:00 That's it. This is the limit. I've been patient for a long time.

SQL is not code. Neither is html. You shouldn't call it a programming language. SQL is a special language which describes a way of working with data. In its pure form it should be avoided as much as possible for developing applications.
211 views13:16
Open / Comment
2021-06-30 10:10:00 It's published!

https://leanpub.com/becomeadataengineer/
66 views07:10
Open / Comment