information here

[Project] PagesXcrawler - A Web Crawler with a Simple GitHub-Powered UI

Hey r/madeinpython! I wanted to share PagesXcrawler, a Python-based web crawler project that I’ve been working on. It uses GitHub Pages to provide a simple, accessible web UI for displaying the information it extracts.

What it does:

• Web Crawling: This tool takes in URLs (formatted as https://example.com:depth) and crawls the specified depth of internal links. It collects data from the pages it visits, making it easy to retrieve structured information.
• User-Friendly Display: The results are displayed in a GitHub Pages-based interface, providing a straightforward view of the crawled data without requiring any complicated setup.

Why GitHub Pages?

I wanted a minimal setup that didn't rely on complex backends, so GitHub Pages serves as a static frontend. GitHub Actions can trigger the crawler as needed, making it an accessible solution for web data extraction.

Feel free to check out the repo and try it out: GitHub - PagesXcrawler

I have created a simple password generator as part of my projects while I learn, This generator creates the password and also shows you the strength of the password

https://github.com/Jean-carje/Secure-Pass-Generator

Link: https://github.com/Dross-Engineering/Encode-Decode

I have created a simple python app which can encode and decode text using a seed.

The encoded text is completely decoupled from the original text meaning that every time you click encode the output will be a different random string with a different length.

This complete decoupling means that to my knowledge the encoded text is completely impossible to decode unless the seed is known.

All versions of the encoded text can be decoded with the original seed so it doesnt matter which one you use.

The app works with all standard English characters including numbers and symbols and preserves new line formatting (\n).

This app is probably far inferior to other solutions out there, and i just built it to see if i could.

I would love feedback and suggestions.

https://preview.redd.it/vzdnba1xdtxd1.png?width=732&format=png&auto=webp&s=06768982add725fd98519335798c9945360e02e5

Just recently joined a project that's done in plotly dash, written by some non programmers and now management wants to scale it, so I've been brought in to clean it up.

I'm just looking for a few projects that take things beyond the minimal examples to help familiarize myself with a concept of a plan for an end state and a path to get there.

I'm posting for a colleague, he's new on reddit and has a post block

Hello! I like scraping with BeautifulSoup, because of its simplicity and ability to perform quick search operations.

However, when more complex selection criteria are involved, it becomes a bit cumbersome, often leading to messy, repetitive boilerplate code.

What started as a simple solution to my own problems has now grown into a full-fledged python package, that I’m excited to share with the community.

soupsavvy, which is BeautifulSoup search engine with clear, intuitive interface, gives infinite flexibility in defining selectors.

You can combine and extend your selectors with ease, which keeps your code clean and maintainable. On top of that, it provides more advanced features like pipelines and object oriented approach.

Let's say, you need to locate `party` element to extract text content from it with BeautifulSoup:

for div in soup.find_all("div"):
    for event in div.find_all(class_="event", recursive=False):
        party = event.find_next_sibling("span", string="party")
        if party is not None:
            break
else:
    raise ValueError("No party, let's go home")

result = party.get_text(strip=True)

With soupsavvy is much simpler, since selection/extraction logic is defined in selector itself. They in consequence can be reused across different scenarios.

from soupsavvy import ClassSelector, PatternSelector, TypeSelector
from soupsavvy.operations import Text

selector = (
    TypeSelector("div")
    > ClassSelector("event") + (TypeSelector("span") & PatternSelector("party"))
) | Text(strip=True)
result = selector.find(soup, strict=True)

Give it a try! Install with pip:

🚀 pip install soupsavvy

For more information, visit:

📚 Docs & Tutorials: https://soupsavvy.readthedocs.io/

💻 GitHub: https://github.com/sewcio543/soupsavvy

I’d love to hear your feedback!

Hey, dev community! 🌟

I’ve been deep into channel automation lately, and I’m excited to share that I just built an API to download TikTok and Instagram Reels and videos effortlessly at cheaper price! 😄 This tool has become a crucial part of my automation workflow, and I think you’ll love it too.

TikTok: TikTok API

Instagram: Instagram Downloader

Youtube: YouTube Downloader

https://github.com/markuslahde/pop-up-ide/

Hi! I published my first PyPI-package today. It enables entering interactive/iterative coding environment from any line of regular python code. Namespace of host environment is copied and can be manipulated inside the environment without irreversible changes. The coding environment has autocomplete and syntax highlighting. Comments and suggestions for improvement are welcome!

tl;dr made a thing, works on my machine

I built a sound level meter and IoT noise monitoring device. It can measure standard acoustical metrics for noise, and transmit them to an IoT dashboard. It is implemented in MicroPython, a Python implementation for microcontrollers (https://micropython.org/).

The sensor node, and the data shown in dashboard

Bit about the implementation:

* Running on ESP32 microcontroller
* For audio input, it uses an I2S digital microphone via the machine.I2S module in MicroPython
* For processing audio efficiently, this uses emlearn-micropython, a Machine Learning and Digital Signal Processing package for MicroPython: https://github.com/emlearn/emlearn-micropython
* For the IoT dashboard, it uses https://blynk.io/

Code and instructions can be found here: https://github.com/emlearn/emlearn-micropython/tree/master/examples/soundlevel_iir

General discussion thread about the emlearn library - where related news is posted: https://github.com/orgs/micropython/discussions/16004

Have you tested out MicroPython or interested in making something with it?

I recently put together an open-source Blackjack Strategy Simulator, and I’d love to get your feedback!

Features:

• 🧠 Basic Strategy Generation: Tailor custom strategy tables based on different rule variations.

• 🤖 Best Move Analysis: Calculate the optimal play for any hand and ruleset, accounting for complex scenarios like splits.

• 💸 Expected Value (EV) Calculation: Evaluate the long-term profitability of your strategies with precision.

• ⚡ Multithreading Support: Simulate millions of hands quickly using multiple cores.

Supports popular blackjack rules: multi-deck, hit/stand soft 17, double after split, surrender, and more.

🌟 Contributions are welcome! Check out the GitHub repo for more details. Don't forget to star it if you like it!

It's up on GitHub, totally free to use: https://github.com/AttackingOrDefending/Blackjack-Strategy-Simulator.

If you check it out, I’d appreciate any feedback or suggestions.

I've had my own small stuffs like youtube video downloader,chatbots,pdf-docx-converot etc.but these are just stupid and random things i made.maybe commenting your projects could give me fresh ideas

If I have a great grip over python from basics to advance,also DSA What specialization would be great to apply these skills

View Poll

How much it time it would take to learn python from basics to advance

Hi all!

I've been working for a client who needed to display code snippets in a Dash app + an easy way to copy them. Since I couldn't find a solution I built one and open-source it. It adds syntax highlighting for the most popular languages.

Check it out here: dash-react-syntax-highlighter

It's pretty basic since I wanted to release something quickly for my client, but let me know if you have any feature requests!

https://i.redd.it/yh73t03wu8sd1.gif

Hi!
I built a new component to display PDFs in Dash apps, hope you find it useful! https://github.com/ploomber/dash-pdf

https://reddit.com/link/1ft1m11/video/rfskeelxgzrd1/player

https://preview.redd.it/xyhqawey4zrd1.jpg?width=1280&format=pjpg&auto=webp&s=413f56ba4fb4867108dbde59c1ec0f6c860b2278

Welcome to our comprehensive Dinosaur Image Classification Tutorial!

We’ll learn how use Convolutional Neural Network (CNN) to classify 5 dinosaur categories , based on 200 images :

• Data Preparation: We'll begin by downloading a curated dataset of dinosaur images, neatly categorized into five distinct classes. You'll learn how to load and preprocess the data using Python, OpenCV, and Numpy, ensuring it's perfectly ready for training.

• CNN Architecture: Unravel the secrets of Convolutional Neural Networks (CNNs) as we dive into their structure and discuss the different layers—convolutional, pooling, and fully connected. Learn how these layers work together to extract meaningful features from images.

• Model Training :  Using Tensorflow and Keras , we will define and train our custom CNN model. We'll configure the loss function, optimizer, and evaluation metrics to achieve optimal performance during training.

• Evaluation Metrics: We'll evaluate our trained model using various metrics like accuracy and confusion matrix to measure its efficiency and robustness.

• Predicting New Images: Finally , We put our pre-trained model to the test! We'll showcase how to use the model to make predictions on fresh, unseen dinosaur images, and witness the magic of AI in action.

You can find more tutorials, and join my newsletter here : https://eranfeit.net/

Check out our tutorial here :  https://youtu.be/ZhTGcw0C3Dk&list=UULFTiWJJhaH6BviSWKLJUM9sg

Enjoy

Eran

Definitely I am not yet a master but I am learning.I will do my best to help.And that will be the point of this community that everyone can help each other.Nobody has to ask a specific person but everyone is there to help each other as a growing yet Relatively new python community of friendly like minded individuals with unique invaluable skill sets! And colabs and buddies!

Hi r/madeinpython

We're stoked to share our latest project with you: Dash MUI. It brings Material UI to Dash, allowing you to create beautiful dashboards without design skills. So far we've implemented:

• Buttons
• Button groups
• Tables
• Accordion
• Card
• Forms
• Grid
• Radio buttons
• Slider

Let us know if there is another component you'd like to see. It's free an open source.

GitHub: https://github.com/ploomber/dash-mui

Demo: https://dash-mui.ploomberapp.io

Dash MUI demo

Hi, r/madeinpython!

I want to present my new library for creating maps with Dash: dash-react-simple-maps.

As the name suggests, it uses the fantastic react-simple-maps library, which allows you to easily create maps and add colors, annotations, markers, etc.

Please take it for a spin and share your feedback. This is my first Dash component, so I’m pretty stoked to share it!

Live demo: dash-react-simple-maps.ploomberapp.io

Demo

Hi all,

Around 75 free spaces left on my functional Python course. You'll learn about list, set, dictionary and generator comprehensions.

https://www.udemy.com/course/functional-programming-with-python-comprehensions/?couponCode=052A7FB50D85A770DB51