Get Mystery Box with random crypto!

Python/ django

Logo of telegram channel pythonl — Python/ django P
Logo of telegram channel pythonl — Python/ django
Channel address: @pythonl
Categories: Technologies
Language: English
Subscribers: 41.09K
Description from channel

admin @workakkk
@itchannels_telegram - 🔥 best it channels
@ai_machinelearning_big_data -ML
@ArtificialIntelligencedl -AI
@datascienceiot - ml 📚
@pythonlbooks -📚books

Ratings & Reviews

3.00

2 reviews

Reviews can be left only by registered users. All reviews are moderated by admins.

5 stars

1

4 stars

0

3 stars

0

2 stars

0

1 stars

1


The latest Messages

2023-07-04 16:37:59
Python Web Scraping

Этот исчерпывающий список содержит библиотеки python, связанные с веб-парсингом и обработкой данных.

Web Scraping: Frameworks
scrapy - web-scraping framework (twisted based).
pyspider - A powerful spider system.
autoscraper - A smart, automatic and lightweight web scraper
grab - web-scraping framework (pycurl/multicurl based)
ruia - Async Python 3.6+ web scraping micro-framework based on asyncio
cola - A distributed crawling framework.
frontera - A scalable frontier for web crawlers
dude - A simple framework for writing web scrapers using decorators.
ioweb - Web scraping framework based on gevent and lxml

Web Scraping : Tools
portia - Visual scraping for Scrapy.
restkit - HTTP resource kit for Python. It allows you to easily access to HTTP resource and build objects around it.
requests-html - Pythonic HTML Parsing for Humans.
ScrapydWeb - A full-featured web UI for Scrapyd cluster management, which supports Scrapy Log Analysis & Visualization, Auto Packaging, Timer Tasks, Email Notice and so on.
Starbelly - Starbelly is a user-friendly and highly configurable web crawler front end.
Gerapy - Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

Web Scraping : Bypass Protection
cloudscraper - A Python module to bypass Cloudflare's anti-bot page.

GIthub

@pythonl
1.2K views13:37
Open / Comment
2023-07-03 19:02:35 10 Useful python scripts

10 интересных скриптов Python.

• Создавайте веб-бота
# pip install selenium
import time
from selenium import webdriver
from selenium.webdriver.common.keys
import Keysbot = webdriver.Chrome("chromedriver.exe")
bot.get('http://www.google.com')
search = bot.find_element_by_name('q')
search.send_keys("@codedev101")
search.send_keys(Keys.RETURN)
time.sleep(5)
bot.quit()

• Улучшение изображений на Python

# pip install pillow
from PIL import Image,ImageFilter
from PIL import ImageEnhance
im = Image.open('img.jpg')
# Choose your filter
# add Hastag at start if you don't want to any filter below
en = ImageEnhance.Color(im)
en = ImageEnhance.Contrast(im)
en = ImageEnhance.Brightness(im)
en = ImageEnhance.Sharpness(im)# result
en.enhance(1.5).show("enhanced")

• Парсинг текстов песен
# pip install lyricsgenius
import lyricsgenius
api_key = "xxxxxxxxxxxxxxxxxxxxx"
genius = lyricsgenius.Genius(api_key)
artist = genius.search_artist("Pop Smoke",
max_songs=5,sort="title")
song = artist.song("100k On a Coupe")
print(song.lyrics)

Получение данных Exif для фотографий
# Get Exif of Photo
# Method 1
# pip install pillow
import PIL.Image
import PIL.ExifTags
img = PIL.Image.open("Img.jpg")
exif_data =
{
PIL.ExifTags.TAGS[i]: j
for i, j in img._getexif().items()
if i in PIL.ExifTags.TAGS
}
print(exif_data)
# Method 2
# pip install ExifRead
import exifread
filename = open(path_name, 'rb')
tags = exifread.process_file(filename)
print(tags)

Поиск в Google
# pip install google
from googlesearch import search
query = "Medium.com"
for url in search(query):
print(url)

Преобразование: шестнадцатеричная система в RGB
# Conversion: Hex to RGB
def Hex_to_Rgb(hex):
h = hex.lstrip('#')
return tuple(int(h[i:i+2], 16) for i in (0, 2, 4))
print(Hex_to_Rgb('#c96d9d')) # (201, 109, 157)
print(Hex_to_Rgb('#fa0515')) # (250, 5, 21)

Конвертация фотографий в формат Cartonize
# pip install opencv-python
import cv2
img = cv2.imread('img.jpg')
grayimg = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
grayimg = cv2.medianBlur(grayimg, 5)
edges = cv2.Laplacian(grayimg , cv2.CV_8U, ksize=5)
r,mask =cv2.threshold(edges,100,255,cv2.THRESH_BINARY_INV)
img2 = cv2.bitwise_and(img, img, mask=mask)
img2 = cv2.medianBlur(img2, 5)
cv2.imwrite("cartooned.jpg", mask)

Тестирование скорости соединения с помощью Python.
# pip install pyspeedtest
# pip install speedtest
# pip install speedtest-cli
#method 1
import speedtest
speedTest = speedtest.Speedtest()
print(speedTest.get_best_server())
#Check download speed
print(speedTest.download())
#Check upload speed
print(speedTest.upload())
# Method 2
import pyspeedtest
st = pyspeedtest.SpeedTest()
st.ping()
st.download()
st.upload()

Проверка состояния сайта
# pip install requests
#method 1
import urllib.request
from urllib.request import Request, urlopenreq = Request('https://medium.com/@pythonians', headers={'User-Agent': 'Mozilla/5.0'})
webpage = urlopen(req).getcode()
print(webpage) # 200
# method 2
import requests
r = requests.get("https://medium.com/@pythonians")
print(r.status_code) # 200

Извлечение текста OCR из изображений
# pip install pytesseract
import pytesseract
from PIL import Image
pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe'
t=Image.open("img.png")
text = pytesseract.image_to_string(t, config='')
print(text)

@pythonl
1.6K views16:02
Open / Comment
2023-07-02 14:30:59
Create a Mock SQL DB in Python from CSV for unit testing

Создание макета SQL-базы данных в Python из CSV для модульного тестирования.

pip install pandas
pip install sqlglot
pip install sqlalchemy
from sqlalchemy import create_engine, text
import sqlglot
import pandas as pd


def execute_sql_query(sql):
query_as_sqlite = sqlglot.transpile(sql, read="postgres", write="sqlite")[0]
mocked_db = create_engine('sqlite://')
pd.read_csv('data.csv').to_sql('table_name', con=mocked_db)
with mocked_db.connect() as connection:
result = connection.execute(text(query_as_sqlite))
return result

@pythonl
2.8K views11:30
Open / Comment
2023-07-02 12:30:11 Machine learning — обучающий для тех, кто хочет погрузится в волшебный мир Нейронауки!

Нейронные сети, машинное обучение, Data Science, изучаем базу, объясняем код, изучаем лучшие проекты, выкладываем бесплатные курсы и книги из области Машинного обучения.

Начать учиться
2.6K views09:30
Open / Comment
2023-07-01 18:01:37
Daily Useful Scripts

Daily.py is a repository that provides a collection of ready-to-use Python scripts for automating common daily tasks.

Daily.py - это репозиторий, который предоставляет коллекцию готовых к запуску скриптов Python для автоматизации обычных повседневных задач.

git clone https://github.com/Chamepp/Daily.py.git

Github

@pythonl
3.0K views15:01
Open / Comment
2023-07-01 16:02:36
https://t.me/backend_architecture

Канал для тех, кто интересуется архитектурой программного обеспечения, оптимизацией производительности систем, масштабируемостью, надежностью и другими аспектами разработки программного обеспечения.

Канал подойдет тем, кто хочет улучшить свои навыки и знания в области backend разработки и system design.

Backend-разработка
System Design
Архитектура
Проектирование

https://t.me/backend_architecture
2.8K views13:02
Open / Comment
2023-07-01 12:17:24 Building a Trading Strategy with Machine Learning Models and Yahoo Finance in Python.

Создаем алгоритм для торговли с помощью моделей машинного обучения и Yahoo Finance на Python.

import yfinance as yf
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
import matplotlib.pyplot as plt

# Step 1: Data Collection
ticker = "AAPL"
start_date = "2021-01-01"
end_date = "2023-01-06"
data = yf.download(ticker, start=start_date, end=end_date, progress=False)

# Step 2: Data Preprocessing
data["Return"] = data["Close"].pct_change()
data.dropna(inplace=True)

# Step 3: Feature Engineering
data["SMA_5"] = data["Close"].rolling(window=5).mean()
data["SMA_20"] = data["Close"].rolling(window=20).mean()

# Step 4: Model Selection and Training
X = data[["SMA_5", "SMA_20"]]
y = (data["Return"] > 0).astype(int)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

pipeline = Pipeline([
('imputer', SimpleImputer(strategy='mean')),
('classifier', RandomForestClassifier(n_estimators=100, random_state=42))
])

pipeline.fit(X_train, y_train)

# Step 5: Model Evaluation
y_pred_train = pipeline.predict(X_train)
train_accuracy = accuracy_score(y_train, y_pred_train)

y_pred_test = pipeline.predict(X_test)
test_accuracy = accuracy_score(y_test, y_pred_test)

print("Train Accuracy:", train_accuracy)
print("Test Accuracy:", test_accuracy)

# Step 6: Strategy Design
data["Predicted_Return"] = pipeline.predict(X)
data["Signal"] = data["Predicted_Return"].diff()
data.loc[data["Signal"] > 0, "Position"] = 1
data.loc[data["Signal"] < 0, "Position"] = -1
data["Position"].fillna(0, inplace=True)

# Step 7: Backtesting
data["Strategy_Return"] = data["Position"] * data["Return"]
cumulative_returns = (data["Strategy_Return"] + 1).cumprod()

plt.figure(figsize=(10, 6))
plt.plot(data.index, cumulative_returns)
plt.xlabel("Date")
plt.ylabel("Cumulative Returns")
plt.title("Trading Strategy Performance")
plt.grid(True)
plt.show()

@pythonl
2.6K views09:17
Open / Comment
2023-06-30 16:01:11 10 Advanced Python Scripts For Everyday Programming

10 полезных скриптов Python для повседневных задач

1. SpeedTest with Python
# pip install pyspeedtest
# pip install speedtest
# pip install speedtest-cli

#method 1
import speedtest

speedTest = speedtest.Speedtest()
print(speedTest.get_best_server())

#Check download speed
print(speedTest.download())

#Check upload speed
print(speedTest.upload())

# Method 2

import pyspeedtest
st = pyspeedtest.SpeedTest()
st.ping()
st.download()
st.upload()

2. Search on Google

# pip install google

from googlesearch import search

query = "Medium.com"

for url in search(query):
print(url)

3. Make Web Bot
# pip install selenium

import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

bot = webdriver.Chrome("chromedriver.exe")
bot.get('[http://www.google.com'](http://www.google.com'))

search = bot.find_element_by_name('q')
search.send_keys("@codedev101")
search.send_keys(Keys.RETURN)
time.sleep(5)
bot.quit()

4. Fetch Song Lyrics
# pip install lyricsgenius

import lyricsgenius

api_key = "xxxxxxxxxxxxxxxxxxxxx"

genius = lyricsgenius.Genius(api_key)
artist = genius.search_artist("Pop Smoke", max_songs=5,sort="title")
song = artist.song("100k On a Coupe")

print(song.lyrics)

5. Get Exif Data of Photos
# Get Exif of Photo

# Method 1
# pip install pillow
import PIL.Image
import PIL.ExifTags

img = PIL.Image.open("Img.jpg")
exif_data =
{
PIL.ExifTags.TAGS[i]: j
for i, j in img._getexif().items()
if i in PIL.ExifTags.TAGS
}
print(exif_data)


# Method 2
# pip install ExifRead
import exifread

filename = open(path_name, 'rb')

tags = exifread.process_file(filename)
print(tags)

6. OCR Text from Image
# pip install pytesseract

import pytesseract
from PIL import Image

pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe'

t=Image.open("img.png")
text = pytesseract.image_to_string(t, config='')

print(text)

7. Convert Photo into Cartonize

# pip install opencv-python

import cv2

img = cv2.imread('img.jpg')
grayimg = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
grayimg = cv2.medianBlur(grayimg, 5)

edges = cv2.Laplacian(grayimg , cv2.CV_8U, ksize=5)
r,mask =cv2.threshold(edges,100,255,cv2.THRESH_BINARY_INV)

img2 = cv2.bitwise_and(img, img, mask=mask)
img2 = cv2.medianBlur(img2, 5)

cv2.imwrite("cartooned.jpg", mask)

8. Empty Recycle Bin
# pip install winshell

import winshell
try:
winshell.recycle_bin().empty(confirm=False, /show_progress=False, sound=True)
print("Recycle bin is emptied Now")
except:
print("Recycle bin already empty")

9. Python Image Enhancement
# pip install pillow

from PIL import Image,ImageFilter
from PIL import ImageEnhance

im = Image.open('img.jpg')

# Choose your filter
# add Hastag at start if you don't want to any filter below

en = ImageEnhance.Color(im)
en = ImageEnhance.Contrast(im)
en = ImageEnhance.Brightness(im)
en = ImageEnhance.Sharpness(im)

# result
en.enhance(1.5).show("enhanced")

10. Get Window Version
# Window Version

import wmi
data = wmi.WMI()
for os_name in data.Win32_OperatingSystem():
print(os_name.Caption) # Microsoft Windows 11 Home

@pythonl
3.0K views13:01
Open / Comment
2023-06-29 13:44:48 10 Tips And Tricks To Write Better Python Code

10 советов и приемов для написания лучшего кода на Python

1) Iterate c enumerate() вместо range(len())

data = [1, 2, -3, -4]
# плохо:
for i in range(len(data)):
if data[i] < 0:
data[i] = 0

# хорошо:
data = [1, 2, -3, -4]
for idx, num in enumerate(data):
if num < 0:
data[idx] = 0


2) list comprehension вместо for-loops

#плохо:

squares = []
for i in range(10):
squares.append(i*i)

# хорошо:

squares = [i*i for i in range(10)]

3) sorted() method

data = (3, 5, 1, 10, 9)
sorted_data = sorted(data, reverse=True) # [10, 9, 5, 3, 1]

data = [{"name": "Max", "age": 6},
{"name": "Lisa", "age": 20},
{"name": "Ben", "age": 9}
]
sorted_data = sorted(data, key=lambda x: x["age"])

4) Хранение данных в Sets

my_list = [1,2,3,4,5,6,7,7,7]
my_set = set(my_list) # removes duplicates

primes = {2,3,5,7,11,13,17,19}

5) Экономьте память с помощью генераторов

# list comprehension
my_list = [i for i in range(10000)]
print(sum(my_list)) # 49995000

# generator comprehension
my_gen = (i for i in range(10000))
print(sum(my_gen)) # 49995000

import sys

my_list = [i for i in range(10000)]
print(sys.getsizeof(my_list), 'bytes') # 87616 bytes

my_gen = (i for i in range(10000))
print(sys.getsizeof(my_gen), 'bytes') # 128 bytes

6) Определение значений по умолчанию в словарях с помощью .get() и .setdefault()

my_dict = {'item': 'football', 'price': 10.00}
count = my_dict['count'] # KeyError!

# лучше:
count = my_dict.get('count', 0) # optional default value

count = my_dict.setdefault('count', 0)
print(count) # 0
print(my_dict) # {'item': 'football', 'price': 10.00, 'count': 0}

7) Подсчет хэшируемых объектов с помощью collections.Counter

from collections import Counter

my_list = [10, 10, 10, 5, 5, 2, 9, 9, 9, 9, 9, 9]
counter = Counter(my_list)

print(counter) # Counter({9: 6, 10: 3, 5: 2, 2: 1})
print(counter[10]) # 3

from collections import Counter

my_list = [10, 10, 10, 5, 5, 2, 9, 9, 9, 9, 9, 9]
counter = Counter(my_list)

most_common = counter.most_common(2)
print(most_common) # [(9, 6), (10, 3)]
print(most_common[0]) # (9, 6)
print(most_common[0][0]) # 9

8 ) Форматирование строк с помощью f-Strings

name = "Alex"
my_string = f"Hello {name}"
print(my_string) # Hello Alex

i = 10
print(f"{i} squared is {i*i}") # 10 squared is 100

9) Конкатенация строк с помощью .join()

list_of_strings = ["Hello", "my", "friend"]

#плохо:
my_string = ""
for i in list_of_strings:
my_string += i + " "

#хорошо
list_of_strings = ["Hello", "my", "friend"]
my_string = " ".join(list_of_strings)

10) Слияние словарей с синтаксисом двойной звездочки **.
d1 = {'name': 'Alex', 'age': 25}
d2 = {'name': 'Alex', 'city': 'New York'}
merged_dict = {**d1, **d2}

@pythonl
3.7K viewsedited  10:44
Open / Comment
2023-06-29 11:44:18
Пройдите тест на углубленные знания в сфере Machine Learning. Professional

Ответьте на 10 вопросов и проверьте, насколько вы готовы к обучению на продвинутом курсе «Machine Learning. Professional» от OTUS и его партнера — Сбера

Время прохождения теста ограничено 30 минут

ПРОЙТИ ТЕСТ: https://otus.pw/8y6P/

Курс доступен в рассрочку.

Пройдете успешно тест, получите доступ к открытым урокам курса.

Нативная интеграция. Информация о продукте www.otus.ru
3.5K views08:44
Open / Comment