Welcome to /r/LearnMachineLearning!

I know machine learning is the future, and as an experienced sw engineer, I’m really interested in it. However, I struggle with math and don’t particularly enjoy it. For example, I tried reading Deep Learning by Goodfellow, but the math felt too complex and hard for me to understand. I have a degree in computer science, but I’m wondering if the ML path is right for me given my challenges with math. Should I start with simpler books, such as Introduction to Statistical Learning? Or maybe at deeplearning.ai ? Can you recommend me other resources?

I love learning by doing and want to really get into ML. I know already some theory and have worked with PyTorch, Hugging face, Scikit learn, langchain, plotly, pandas and Numpy. Now I really want to create an actual useful project which is not just some cookie cutter tutorial project. I already followed some cookie cutter tutorial projects with using YOLO for object detection and also one with BERT from huggingface, however these projects were basically like “take this ready made model and train it with data and you’re done” which was really boring.

I’d like some projects where I have to go deep into AI and not just using Someone’s premade model with my own data. What recommendations do you have?

Hi everyone, I am a software engineer with 1.5 years of experience in a service-based company. For the first year, I primarily worked on deep learning and LLM-related projects. Over the past six months, I have been focusing on full-stack development while also working on DL/LLM-based PoCs on the side for my company.

I am now looking to start my journey on Kaggle and would greatly appreciate any tips or advice. Thank you!

Hi all, I’m working on a research paper and would love your opinion on this! When visualizing data distributions, do you think it's better to separate the KDE and the histogram into two different plots or overlay them on a single plot? would separating them make the visualizations harder to follow? I’d appreciate any insights especially if you’ve tackled this in your own work! Thanks in advance for your thoughts!

Hi everyone!

I've been selected to participate in an AI and Cybersecurity Hackathon, and the group I'm in focuses on AI for DNS Security. Our goal is to implement AI algorithms to detect anomalies and enhance DNS security.

Here’s the catch: I have no prior background in cybersecurity, and I’m also a beginner in applying AI to real-world security problems. I’d really appreciate some guidance from this amazing community on how to approach this challenge.

A bit more about the project:

Objective: Detect anomalies in DNS traffic (e.g., malicious requests, tunneling, etc.).

AI tools: We’re free to choose algorithms, but I’m unsure where to start—supervised vs. unsupervised learning?

My skillset:

Decent grasp of Python (Pandas, Scikit-learn, etc.) and basic ML concepts.

No practical experience in network security or analyzing DNS traffic.

What I’m looking for:

1. Datasets: Any recommendations for open-source DNS datasets or synthetic data creation methods?

2. AI methods: Which models work best for anomaly detection in DNS logs? Are there any relevant GitHub projects?

3. Learning resources: Beginner-friendly material on DNS security and the application of AI in this domain.

4. Hackathon tips: How can I make the most of this opportunity and contribute effectively to my team?

Bonus question:

If you’ve participated in similar hackathons, what strategies helped you balance learning and execution within a short timeframe?

Thank you so much in advance for any advice, resources, or personal experiences you can share! I’ll make sure to share our project results and lessons learned after the hackathon.

Hi.
I am trying to launch a chatbot using llama 3.1 and langchain. Huggingfacepipeline is from langchain_huggingface library. I am generating the text using llm.invoke(prompt) method. When I run the code, it gives Setting `pad_token_id` to `eos_token_id`:None for open-end generation. and the output is not stopping even when I set max_new_tokens to a high number. Can anyone explain to me why the output is not stopping

model_path = 'hf_llama/llama_8B'

def load_model(model_path):
    tokenizer = AutoTokenizer.from_pretrained(model_path)

    model = AutoModelForCausalLM.from_pretrained(
        model_path,
        device_map='auto',
        torch_dtype=torch.float16,
    )

    pipe = pipeline(
        'text-generation',
        model=model,
        tokenizer=tokenizer,
        torch_dtype=torch.float16,
        device_map='auto',
        do_sample=True,
        top_k=50,
        top_p=0.9,
        num_return_sequences=1,
        eos_token_id=tokenizer.eos_token_id,
        # max_length=400,
        temperature=0.9,
        max_new_tokens=500
    )

    llm = HuggingFacePipeline(pipeline=pipe)
    return llm

As a student from life sciences, I'm interested in the field of AI and MLand want to learn (I've even started learning python) . But I want to know is it difficult to learn ML from any online resources, specially for a guy who studied maths only till lower secondary?! Also if u can suggest some good resources it'll be very helpful!

Apart from rags what are the bother gen ai projects to consider doing abd adding on my resume ?

Hey guys, I'm an EE undergrad student and would like to take up some ML topic for my thesis and take applied ML for masters. I'm studying in Poland. I would like to know if there are many opportunities in ML on the market or I should shift to other branches

suppose i created a model to detect if an image has a handwritten digit 4, if i pass the model a completely white image, each pixels of value 255, will the model classify it as a positive case of the digit 4?

I was thinking that even though the model is detecting an image with a handwritten digit "4", the model would have parameters of value = 0 for irrelevant parts of the data and a non zero value for relevant parts of the data.

For the irrelevant parts, 0 (the value of the parameter) x 255 (the pixel value for the white image) would be 0 which would also be the case if we were to give an image with a handwritten digit of "4". For the relevant parts, the values/result would be the same for both the white image and the image with handwritten digit "4" because at that relevant parts of the image (based on the filter), the pixel value would be or close to 255 because that region would be white.

Hence adding them up (or doing dot product), won't the values after doing the dot product of the parameters (weights) and the completely white image be the same as that of the image with handwritten digit of "4"?

Guys i am trying to decide between the two paths and i must pick one at the moment. Can you please tell me which tasks do they do on the job? The poi t is that i like coding basic stuff like a library or a to do list (but in C, not a website). I dont know if software engineering is like this on the job as well.

If you have any advice to help me decide please feel free to share it!

Hi, I recently asked about a PC I am planning to put together for ML. I just need some advice on which case to get I'd prefer a quiet machine, but without compromising airflow and performance. Here is the machine spec:

• CPU: AMD Ryzen 9 7900 3.7GHz 12 Core CPU - 5.4GHz Turbo
• Motherboard: Gigabyte B650 Aorus Elite AX V2
• GPU: NVidia GeForce RTX 4070 Ti SUPER 16GB
• Memory: 64GB Corsair Vengeance DDR5 5200MHz CL40 (2x32GB)
• Primary SSD: 2TB WD BLACK SN850X NVMe PCIe 4.0 SSD (7300MB/R, 6600MB/W)
• Secondary SSD: 2TB WD BLACK SN850X NVMe PCIe 4.0 SSD (7300MB/R, 6600MB/W)
• CPU Cooling: Noctua NH-D15 Chromax Pure Black CPU Cooler
• PSU: Corsair HX1000i 1000w 80 Plus Platinum Fully Modular ATX 3.1.
• WiFi: 5800mbps Gigabyte GC-WIFI7 6GHz Wifi 7 Bluetooth 5.3.

I am undecided between the following cases:

• Be Quiet! Pure Base 500, DX. Pro: seems to have better airflow than other Be Quiet cases. Con: only one front USB-A port.
• Corsair 4000D Airflow. Seems similar to the Be Quiet! Pure Base 500DX, and has good reviews.
• Be Quiet Shadow Base 800 FX. Similar to the Base 500 DX, but it has an additional USB-A port and might be more suitable for a high performance build, especially in light of future upgrades.

All three cases are well within my budget. I'd be very grateful for any advice, also about possible alternatives.

I'd be grateful for any advice.

I'm confused on how I'm supposed to choose how much % of my Convolutional layers I should prune (structured pruning)? Doing grid search/random search/bayesian optimization take soooo long so I want to make sure before I run anything (I'm training on MNIST, CIFAR-10 and Imagenette). What's the standard protocol?

There's a lot of papers that I've read through that give the theoretical framework but I'm looking for some specific workflow/algorithm. I'm working with Pytorch's prune.ln_structured()-method.

Thank you so much for any responses.

Hello everyone, i am a student in second year of cs and i have taken an interest in Ai, ml engineering. I would like to ask for best course on udemy for learning ai ml engineering that can help me understand the concepts, math etc and help me land a job in big tech companies. Reason why i choose udemy is because in my country accessing udemy is easier then coursera. Thank you for your support!

I have a dataset of shapes drawn on paper by students (circle, square, diamond, oblong, and rectangle only). Both actual and digital (scanned into black and white). My Goal is to train model that could simply detect such objects in an image.

I have around 4000 images of different sizes amd resolutions. I am using YOLOv5M and currently achieves 96%mAP on test set.

I labeled the dataset on Roboflow and applied their preprocessing options, specifically:

Auto orientation

Resize to 800×800

Greyscale

Adaptive Histogram

I wanna achieve > 98 mAP, but cannot do it even after many times training. I also did diff. hyperparameters but sometimes it just gets worse. So I'm wondering if I could do something more with my data.

Any tips on what preprocessing or augmentations I should perform regarding my case?

Hello! I've been working on a site that collects reviews about online courses and summarises them - using LLMs.

The first course i did it for is FastAI : Practical Deep Learning for coders.

• Can i get some feedback on what you think? https://coursereviewcollector.com/fastai
• And any feedback on the site itself would be great https://coursereviewcollector.com/
• Let me know if there is another course you would like me to do this for as well! I'm building it out the site now so I'm looking for suggestions

Can you guide me a little.

Hii, I have been looking for email address of people that could help me get a job or a referral for DS or ML jobs in US, I am basically looking for cold emailing. Can you suggest me where should I look for their email addresses?

Hey everyone,

It’s pretty obvious that AI agents are the future—they’re already transforming industries by automating tasks, enhancing productivity, and solving niche problems. However, I’ve noticed a major gap: there’s no simple, centralized marketplace where you can easily browse through hundreds (or thousands) of AI agents tailored for every need.

I’ve found ones like: https://agent.ai/, https://www.illa.ai/, https://aiagentsdirectory.com/, https://fetch.ai, obviously ChatGPTs store- however I think there’s potential for something a lot better

Imagine a platform where you could find the exact AI agent you’re looking for, whether it’s for customer support, data analysis, content creation, or something else. You’d be able to compare options, pick the one that works best, and instantly get the API or integrate it into your workflow.

Plus for developers: a place to showcase and monetize your AI agents by reaching a larger audience, with built-in tools to track performance and revenue.

I’m exploring the idea of building something like this and would love to hear your thoughts:

• Does this resonate with you?
• What kind of AI agents or must have features would you want in a platform like this?
• Any pain points you’ve encountered when trying to find or use AI tools?
• Any other feedback or considerations?

Let me know what you think—I’m genuinely curious to get some feedback!

https://preview.redd.it/k01lc8iugg5e1.jpg?width=1280&format=pjpg&auto=webp&s=303964f172945fd164a2b56e0a4eabb4bc4d7711

👁️ CNN Image Classification for Retinal Health Diagnosis with TensorFlow and Keras! 👁️

How to gather and preprocess a dataset of over 80,000 retinal images, design a CNN deep learning model , and train it that can accurately distinguish between these health categories.

What You'll Learn:

🔹 Data Collection and Preprocessing: Discover how to acquire and prepare retinal images for optimal model training.

🔹 CNN Architecture Design: Create a customized architecture tailored to retinal image classification.

🔹 Training Process: Explore the intricacies of model training, including parameter tuning and validation techniques.

🔹 Model Evaluation: Learn how to assess the performance of your trained CNN on a separate test dataset.

You can find link for the code in the blog : https://eranfeit.net/build-a-cnn-model-for-retinal-image-diagnosis/

You can find more tutorials, and join my newsletter here : https://eranfeit.net/

Check out our tutorial here : https://youtu.be/PVKI_fXNS1E&list=UULFTiWJJhaH6BviSWKLJUM9sg

Enjoy

Eran

#Python #Cnn #TensorFlow #deeplearning #neuralnetworks #imageclassification #convolutionalneuralnetworks #computervision #transferlearning

Someone descent with ML and knows python. Also a known with bit of c++ n java

Regardless with an industry / job / interview perspective what should a person practice DSA in ?

Python / Java / C++ …..?

I have both the copies physically, so that is not the problem.

Hi there, other Redditors!

Like many of you, when I first started working in the AI field, I wanted to build some basic Machine Learning models from scratch in order to better understand how each algorithm works, improve my programming and math skills, or simply produce an eye-catching, difficult project to put in the résumé.

After spending some time searching for resources that could help me guide my studies, I discovered that the majority of scratch implementations that are currently available are either i) outdated (having been implemented years ago using Python 2 or an earlier version of Python 3); ii) too difficult to understand (using a lot of difficult, unfriendly optimization techniques or with poorly written code); or iii) too simple (only covering binary classification).

With that in mind, I made the decision to develop user-friendly, uncomplicated, organized, and simple implementations from scratch. Aside from all of that, I've always wanted to create an open-source project so that others, particularly novices and those with less than a year's experience (like me), can collaborate with others, contribute to public projects, and experience Git firsthand (some of these implementations were made by other contributors!).

Here are some implementations that are available:

• Algorithms (Random Forest Classifier and Regressor, Decision Tree Classifier and Regressor, KMeans, KNN Classifier and Regressor, Gaussian Naive Bayes, Linear Regression, Logistic Regression, PCA, Perceptron, MLP Classifier and Regressor, SVM Classifier and Regressor);
• Regression and classification metrics;
• Distance metrics (such as Euclidean);
• Data split functions (such as KFold);
• Activation and loss functions;
• Scalers (such as MinMaxScaler) and encoders (such as One Hot Encoder);
• and a few things more!

Project's link: https://github.com/rafaelgreca/scratchml

Disclaimer: The goal of this library is to provide code that is simpler, easier to understand, and more approachable for artificial intelligence enthusiasts and beginners who want to contribute to an open-source repository or who want to learn more about how algorithms work. It is not meant to replace existing libraries that are better, more optimized, and have a wider variety of implemented algorithms (such as scikit-learn, PyTorch, Keras, and Tensorflow). If you want to use optimized implementations with accurate results, please use one of the previously mentioned libraries.

P.S.: I accidentally deleted the other post, so I am posting again. :-)

So, I gave an exam and in it one's answer was high f1 score implies low bias. But I am thinking about a scenario in which, a dataset with 9000 positive examples and 200 negative examples. If I train a stupid model which only predicts positive class for any data point. wouldn't that model have high bias as well as high f1 score. One person told me this model actually has low bias. so I am confused.

Hi guys

I have ten years of work experience in firmware development for 5G but I want to move my career to ai and ml related development. I have started with Andrew ng specialization course on machine learning and while the content is good theoretically there isn't good enough practical learning on that course. Can someone suggest small exercises and projects to practice with the course to develop on hands skills for different techniques like regression and classification. Want to practice and create my own repository of what I have done to build a strong foundation.

Thanks in Advance

Whether it’s a big breakthrough or a fun side project, if it touches ML, it belongs here. Feel free to share what your project does, how it works, and don’t forget to drop a GitHub link if you’ve got one. Let’s inspire and learn from each other!

I have finished the first two courses of Andrew's ng machine learning specialization, and I still haven't done any personal projects, and the doctor needs to see a demo of our project in less than a month i'm going to be handling the machine learning part in the project and i need to create a recommendation system for courses , path generating model based on user's preferences and automated grading system for MCQ quizzes and a chatbot (can use APIs ) , please tell me where to start and how to organize these problems.

thanks.

We all saw in class the trade off between bias and variance, that we don't want our train loss to keep going down and our test loss go up.

But in practice I feel like doing hyperparameter tuning for classic ML models with GridSearchCV / BayesSearchCV is not enough. Even though I do cross validation, the search.best_model obtained at the end is almost always overfitting.

How can you actually perform a search that will give you a robust generalized model with higher chances ?