/r/Backend
For back-end programming discussion.
Post news, information, tricks, tips, and techniques related to backend engineering.
Jobs, advertisements, and off-topic posts will be removed.
/r/Backend
Hello,
I am a backend engineer with 10 tears if experience in the industry.
I am getting a bit bored with the recent trends in the backend world, while also starting to think about entrepreneurship.
It feels like almost every new startup is doing AI and I know nothing about it, and would like to know it better.
Where would you start from?
Hello! I’m looking for a service that would allow me to easily cache third party APIs. I can’t manage to find any, which I find strange because I think my usage is quite simple.
Suppose I’m making a website with pages for each major city in the world. Now, on these pages, I have a dynamic section that fetches an API to get a list of concerts. Currently, each time an user accesses one of these pages, a request is made to get the concerts for the city. But this is wasteful because if 200 people look at the London webpage on the same day, 200 requests will be made to the third party concert API. So that’s the problem.
The solution I imagined would be a simple server that would intercept my requests, perform the corresponding request to the third party API, cache the response for a day, and then only serve the cached response during that time.
For example, my request currently looks like this:
GET
https://concertapi.example/london
And I’d like it to become:
GET
https://cache.mydomain.com/concerts/london
Simple right?
I could imagine this service providing basic configuration, like what API should be called on the concerts endpoint, but then it would simply get the rest of the url and pass it as url params to the API. Same for caching, I would like to be able to define the caching duration per endpoint.
It could be self-hosted, it could be a Saas, custom domain option would be a nice to have but that’s not a requirement… I’m just looking for a solution.
If that doesn’t exist. Any idea why?
And so, how hard to build would it be? I can imagine a simple Node.js server backed by a Redis database…
I’m open to any idea. Thank you for your help!
I'm currently working on enhancing a Laravel project, which involves three environments: Development, Stage, and Production.
My workflow typically starts with coding locally in the Development environment. Once I'm satisfied with my changes, I create a pull request to merge into the Stage environment. After merging, our QA team tests the changes. If any issues are found, I address them and create a new pull request for the Stage environment, repeating the process until everything is stable.
In addition to this, we have our own packages in Laravel, which I develop separately. I update these packages on the Stage environment as needed.
However, I've noticed that I'm spending a significant amount of time creating pull requests, particularly due to multiple rounds of testing and review. Finally, when the changes are ready, I create a pull request for the Production environment and request a senior colleague to review before merging.
As a developer, I'm seeking a more efficient approach to reduce the time spent on creating pull requests. Can you suggest any fixes or improvements to streamline this workflow?
So, I want to start my own free-to-use business-in-a-box infrastructure, as I'm too stubborn to fork over my own money to microservices to handle processing and all that dumb stuff. I want to handle everything from scratch with my own code, such as handle, transactions, billing, inventory, etc. from scratch. Any suggestions on framework, how to start, what to do, or anything else?
Hello there,
Has anyone used pocketbase, appwrite or supabase for a customer project here? If so, what type of project was it? Why did you choose this tool? We can talk about it in PM if you agree.
Thanks
I am a CS Major student looking for internship in Node.js. My goal is to get internship or fulltime job.
I learnt Node.js and applied to various internship positions out of hundreds of applicant per job. I think Nodejs is really saturated. I made good Node.js projects but It is so hard to get noticed among hundreds of applicant. It doesn't matter how good I am, I won't be hired.
So I decide to learn another language and make projects. Which Backend Programming Language is easier to get a Entry Level Job with relatively low applicants and higher acceptance rate?
Python(Django / Flask), Java(Spring) , C# (ASPNET), or GoLang?
I am currently a Site Reliability Engineer, but I am interested in transitioning to more backend-focused roles that involve coding. Could you provide some advice on how to prepare for these positions and where to begin?
I started with learning FastApi and Django since I know Python. I also know a bit of golang.
app.patch("/task/:id/edit", async (req, res) => {
const { id } = req.params
await Task.findByIdAndUpdate(id, req.body, { runValidators: true, new: true })
res.redirect("/task")
})
I am unable to render the updated task. Is it because I did not pass the updated task? Shouldn't the DB update automatically? I am able to redirect but the task is not updated.
Hey guys, I’m a backend engineer and I have never gone deep into cache.
Imagine that you have a query with lte and gte parameters and you cache it. Then, when a user modifies an entity that is inside that query you need to delete/modify that cache entry, but the key is built with the params of the query, so you don’t know the exact key.
How do you solve this problem??
My first approach was to get the redis keys by pattern and then do the deletes, but when you have a redis cluster, you would have to get them from all the nodes (it is scaled horizontally and each node saves a portion on the keys), so this can have a bad performance.
Want to know where to start, I want to learn python and rust and C#, what are online courses or platforms I can learn these?
I can also read online books if u can have suggestion Free code camp is one I have thought but haven’t tried
Thanks
I have began backend development and I am also a competitiive Programmer on codeforces and leetcode and I aspire to get an internship or a full time job soon in backend development. Can you suggest some good projects that uses good algorithms that I can make to stand out in my resume.
I am currently doing backend using nodejs
I am currently making API for a project in Spring Boot and there are currently separate API for search and filter. The filter API also has the sort by and sort order in it as well. Both API's have Pagination.
But I was thinking what if someone tries to search on the filtered data, then that will not work like expected as a seperate API will be triggered for search.
I need help with this. So, will it better to combine both the API's together OR should I take the keyword as a parameter in the filter API and use it further filter the API results and let the search be a seperate API.
The search API works on only one the columns in table.
i would like to learn/collaborate with someone who has passion in backend,personally i am familiar with front end and trying to develop some little projects,could team up later
Hey,
I have mentored various people while working in my workspace. I can find few hours time on Saturday and Sunday, so I am thinking of helping new software engineers/ or students studying to help them become successful in their path.
I am working as a Senior Software Engineer (Backend) having 6 years of development experience.
Let me know if anyone is looking.
Edit: I am getting many chat request which is a good. I will reply back it may take time.
I want to know how authentication works on a real life project rather than personal projects.
In my personal projects - I store the token inside user's cookies then verify it that's it.
I want to know the types of authentication that are used in real life like session based authentication.
Background: Crafting BDD-style test scenarios within Postman can pose a significant challenge.
Solution: The design offers a solution by enabling the creation of standalone test scenarios for your requests. This allows for seamless modification of request bodies, headers, and parameters independently with each scenario.
Does this make sense?
A few years ago I had a side project and now I am considering to continue its development.
The project is called Boxes and will allow Backend developers to deploy generic services easily, for example user-auth, blob-storage, localizaton, payments, webhooks, etc...
The project is a docker compose you can run and have an admin site, the admin site have a "store" of backend services you can deploy in one click.
Every service in the "store" is a bundle of docker image and an openapi schema.
When you click install/deploy the project register the openapi schema at the api gateway and runs the docker container.
Because every request is going through it, the api gateway also create logs and statistics that are displayed in the admin site.
In the admin site we can find a new tab for the deployed service with client code for it (generated using the openapi schema), logs and statistics.
The data layer will contain databases and caches so we could pass their urls/hostname into environment variables of other services.
In the past I had created the admin-site, api-gateway, sdk generator, containers management, but didn't get to a publishable state.
What do you this about the Idea? Will you use something like that?
Its like other BaaS but much more modular, open and extendable.
Right now we use NodeJS for our backend because it’s all I knew moving from frontend to backend at the time.
Curious what languages are objectively the best for shipping fast and cutting the fluff out of building a backend?
We’ve got a public and private API, mongodb, bajillions of HTTP requests, and lots of data manipulation, serverless (so cold start times are important).
MOST of the code is helper utils and NPM packages.
Wondering what the better way is.
Sorry if this isn’t the right place to ask this
What are the most appropriate or reasonable reasons to resign as a backend developer? Except for lists below here.
ME:
Need help for my mini project. My project is parking lot management system, where users can book a slot in advance and I want to give user notification by mail when the user's parking time is to expire ( within 10 mins) how can I do this..please help or provide some resources. It will be much helpful please help
I'm fairly new to backend and had just learned some basic CRUD on django. To build a social media site like Twitter with chat function, which is more suitable, Django or Supabase?
Hello! I work as a solo dev on a rather write-heavy backend. Naturally, I want to use database transactions so if something fails, it does so predictably and reverts. While it is easy to do so on simple endpoints, procedures consisting of lots of queries/logic and sub-functions are a mess.
I use Postgres, and my ORM supports both interactive transactions (with logic in between queries) and ? transactions (waiting for multiple queries at once, run inside a tx).
A sample "complex" endpoint looks like this:
- A value is read from database, some cursor is increased. I have a global lock in Redis, so there are no race conditions with the cursor/value
- Some work is made based on the value and request data. This spans multiple smaller functions and is a mixture of logic, queries, and sometimes other rather slow IO. Sometimes it's a 3rd party API request, sometimes fetching data from cache (Redis) or calculating something more expensive. Sometimes it requires a sub-procedure that's an interactive transaction in itself.
- If request fails, an attempt to revert the cursor is made
Ideally, I want the entire request to either succeed or fail, and everything to run without the need of a lock. But running everything in a single interactive transaction is almost impossible due to reference passing clunkiness and long-running stuff not related to the database (which should not be run inside a transaction).
How do you handle this in your applications? Do I overthink it, and having multiple transactions in a request is fine? I rarely had problems with this approach in production, but it feels wrong.
Hi I’m a fresh graduate from computer science i was searching for some internships and i get an offer from placement dost company, does anyone has an experience with them?
Maybe it’s not good as companies that offers internship with tutor but i didn’t get any so it’s better than nothing.
If anyone has any recommendations about internships (remote) pls let me know.
There’s popular mern stack by the internet but i don’t want to follow the crowd and I seen posts online and seeing that it mainly for web dev(correct me if wrong) so seems like not that great.
Going for either python or rust, maybe both though
Hello everyone, I want to start learning backend dev, and currently I'm in CSE 3rd year. So, it would be a great help if I could get a simple roadmap or resource to start learning. The roadmap on roadmap.sh is too extensive and overwhelming, and I know only basics of frontend dev, and I am not interested to continue with it, while the site almost covers everything a lot of frontend dev.
Hi, currently I'm building a full-fledged chat application using Node.js. I'm a frontend developer with 3 years of experience in React, and I'm looking to transition to the backend. Ihave intermediate-level knowledge of writing Express endpoints, authentication, and some database queries. In order to push my limits, I'm looking to build a full chat system with Node.js that includes user and group management, multimedia capabilities, and Socket.io connections. However, I need help understanding how to properly design this chat system, including the use of message queues. Can anyone help me understand and provide guidance on how to develop this application? I'm seeking assistance in designing this chat system so I can gain experience in various backend concepts, and perhaps others can use this application if needed.