I'm a total newbie when it comes to AWS and web hosting. I've built a simple static website (HTML, CSS, and JavaScript) and I'm eager to get it online. I've been trying to figure out how to host it on AWS S3 but I'm completely overwhelmed.

I've created an S3 bucket and uploaded my files, but I'm not sure how to make it accessible to the public. I've tried messing around with bucket permissions and policies, but I keep getting errors.

I have a nextjs app with mongoDB that is hosted to Vercel as it's still in play stage.

I want to move to aws for a better cost optimization, but I'm not sure how to do it.

I still want to take advantage of the serverless api routes that vercel offers out of box. I also want to introduce websockets for live data updates on some components.

I thought of Amplify and AppSync but I'm not quite familiar with it. I also thought of making the apis to lambda functions but I'm not using dynamodb and I think that will overload the database connection.

Any suggestions or tips, from host to serverless apis and live data and costs are welcome.

I set up an old AWS account years and years ago and ended up not using it and thought I had deleted everything. Apparently there's a backup I've been getting charged for every month for the last 5 or so years so I'd like to cancel it but am failing to find where to manage it.

These are the charges: https://imgur.com/m9WvDlH

My RDS resources summary: https://imgur.com/Xlj6hfg

Does anyone know where I can go to cancel this? I have gone through every snapshot, backup, etc page I can find via the UI and the search but I cannot for the life of me figure out where to manage this.

Thank you for any help.

Edit: thanks for the insanely fast help! The issue was I was looking in the wrong region. Never knew that mattered. Maybe I should finish those AWS courses.

I'm trying to handle auto-responses with AWS SES.

The general docs explain:

Auto response – The email is accepted by the ISP, and the ISP delivers it to the recipient. The ISP then sends an automatic response such as an out-of-the-office (OOTO) message to SES. SES forwards the auto response notification to the sender.

And

Make sure that your SES-enabled program does not retry sending messages that generate an auto response.

The notification schema docs explain:

Transient - General
The recipient's email provider sent a general bounce message. You might be able to send a message to the same recipient in the future if the issue that caused the message to bounce is resolved.
Note: If you send an email to a recipient who has an active automatic response rule (such as an "out of the office" message), you MIGHT receive this type of notification.

I obviously do not want to resend an email to a recipient with an OOTO, as that would generate another soft bounce (Transient-General), and I'd resend, and so on.

The explanation above states "might", so it's not foolproof. How do I detect OOTO?

I have done built some RAG project but never using AWS’s tools. I would like to try some of them to see if I can somehow make it more efficient and add features such as guardrails which I never did. I do not know where to start though. Can you recommend me how to start and approach it?

So I have an application load balancer which routes requests to my application ECS tasks. Basically the load balancer listens on port 80 and 443 and route the requests to my application port (5050). When I configured the target group for those listeners (80 and 443), I selected IP type in the target group configuration but didn’t register any target (IP). So what happens now is, if any request comes in from 80 or 443, it just automatically register 2 IP addresses (Bcus I am running two task on ECS) in my application target group registered targets. I have a requirement now to integrate socket.io and in my code, it’s on port 4454. When I try to edit the listener rule for 80 and 443 to add socket target group so it also routes traffic to my socket port (4454), it doesn’t work. This only work if I create a new listener on a different protocol (8443 or 8080) but it doesn’t register IPs automatically in the registered target in socket target group. I manually have to copy the registered IPs that are automatically populated in the application target group and paste it in the socket target group registered targets for it to work. This would have been fine if my application end state doesn’t require auto scaling. For future state, So when I deploy those ECS tasks in production environment, I’ll be configuring auto scaling so more tasks are spinned up when traffic is high. But this creates a problem for me as I can’t be manually copying the IPs from the application targets group to socket target group just in case those tasks grow exponentially when traffic is high. I would want this process to be automatic but unfortunately my socket target group doesn’t register IPs automatically as my application target group does. I would be really grateful if someone can help out or point out what I’m doing wrong

I'm new to AWS and have been exploring its services, particularly those offered in the free tier. I've also looked into dedicated GPU rental services like Vast.ai, Runpod, etc. I'm considering an arbitrage strategy: renting a GPU instance from AWS or another major cloud provider and then listing it on these marketplaces for profit. AWS's initial $200 free credits could help kickstart this venture.

Here are my main questions:

1. Is this allowed under AWS's Terms & Conditions?
2. How practical and profitable is this approach?
3. How can I minimize costs while the instance is not actively rented? I want to avoid wasting money on an idle instance.

I'd appreciate any insights, tips, or experiences you can share. Understanding the feasibility of this idea and any potential pitfalls will be incredibly helpful. Thank you!

P.S. If there's a more suitable subreddit for this question, please point me in the right direction.

Hey ya'll. I have a SDE interview for a L5 role in the AWS team. My phone interview with the HM would be next week. I am told there would probably be leetcode questions/behavioral questions. Does anyone have recent experience in this? What sort of questions should I expect? Will there be any technical questions outside of the leetcode ones? Any tips on how I can prepare myself in these last few days would be greatly appreciated! I am not sure if this is the correct platform but will also post on r/leetcode.

The role is based in the US and I am recent college grad, so I am definitely very scared.

https://preview.redd.it/qcugtkrr8kyd1.png?width=2484&format=png&auto=webp&s=30c8f13a28f69a5f18b9dc125570494e0d239fd8

I know I can do all this with a Linux VM, but I don't want to install a Linux VM right now. The whole point of cloud is so that we don't need to use our personal devices, isn't it?

Instead of having a nice selection of layers ready for us to use, AWS makes us do this! I want to use statsmodels, which is lightweight enough for Lambda (under 250MB).

So I create a nice CloudFormation template so that it can be IaC. I would create an EC2 in a linux distribution, and then package the required layer, and then store the layer on S3. Then, I would download that zipped layer to use on Lambda. This is already way too much effort for cloud tbh, why should customers have to do this all by themselves?

But fine, I put the CloudFormation template in, and fix a bunch of weird errors that it gave me. Then, it asked for a layer bucket name (I wrote it already in CloudFormation by the way) and then a key, that I have to go generate on the EC2 page. Ok, then it failed because the bucket has already been created.

So I go back to S3 to delete that bucket I just made for this template. Then, I run it. Oh guess what, failed again. Why? AMI does not exist.

Wouldn't it make more sense to have something in Lambda that I can just type in what I need and it would generate it for me, and I can put it into the layers? I see that they have finally made ONE single useful layer for my uses, AWSSDKPandas-Python312, but I guess if we want to have other layers, we have to be professionals or have a ton of time to kill to make these custom layers ourselves.

I give up. This is extremely stupid and completely user-unfriendly. Why are they spending time developing a bunch of useless services that no one will ever use and not giving us these basic functionalities?

When a user uploads a doc, it chunks, embeds, then persists it. This currently takes 25 seconds with multithreading on 6 cores for me. The chunk size is small (250), so there's a lot to process. I need this to be faster, and I need to add support for concurrent file uploads.

Here's my current setup:

embeddings: openai small embedding model (batch embed)

chunking: langchain recursive text chunker (250 chunk size)

persistence: Mongodb vector store (batch persist)

I see several companies online seemingly able to upload & embed my same 200 page document in 6 seconds (chatgpt), and in parallel so i'm wondering if i'm missing something that everyone else is using. What tools libraries/tools do you guys use to have fast chunking, embedding, & persistence?

My current idea is to achieve fast parallel uploads via a separate AWS lambda invocation per upload, and have a high amount of cores per lambda so we can achieve paralell processing of chunks in the lambda. Thoughts/advice?

Hello all,

I have been sending payloads to my S3 via IoT Core MQTT into S3 every 1 minute interval.

Is there a service in AWS where I can actually aggregate my 1 minute payload before sending it into my S3 bucket?

For example, I would like to collect 15 payloads (which is the same as 15 minutes) and then send a single JSON file into my S3.

Any advice is greatly appreciated!

I regularly send out a chunk of emails to my newsletter subs and the list is about 24k-ish. I have noticed really low opening rates and conversions recently and saw in my AWS SES panel it says that I am sending about 13k-ish on my mailing days.

Is there a normal reason for this discrepancy?

Could this be because I am sending them too quickly? My aMember customer portal doesn't make it clear how the limiting works.

Anyone have any experience in this sort of thing?

Hi, I'm planning to use Nitro enclave for secure processing of sensitive data. The proposed flow is like this:

1. user needs to send sensitive data in an encrypted form to parent ec2 instance
2. ec2 instance should forward that data to nitro enclave
3. enclave should be able to decrypt that data, make some processing and return data to parent ec2 instance in an also encrypted form
4. parent ec2 instance should return that data to user.
5. user should be able to decrypt that data locally

My question is, what type of cryptographic approach should I choose to implement this type of communication? If it's about KMS, should I use data keys somehow? I struggle to imagine the flow with these keys because I want this user-passed-data to be only encrypted and decrypted on user side and inside enclave (not in the ec2 parent instance). Thank you.

Warning to all - we had about 25 lightsail instances, average size 4GB, all with auto snapshot on. So maybe 150GB stored, at $0.05/GB - that's $7.50. (some manual stored snapshots would push that to about $10/month)

Our bill was $100/month - for snapshots!. Even when I pulled almost all the instances off, shaved the stored snapshots down - our snapshot cost ROSE.

They say we have 1.6TB of snapshots!

I just looked at the bill in detail. I wonder how support will deal with this.

Currently working as System Designer in Metro Signaling system which is not into software related field. But i have masters in Big Data Analytics and I'm planning to get "AWS Solutions Architect - Associate" certification in next 2 months. Also, I'm planning to be sharpening my System Design skills.

PS: I'm from India.

So, what type of job roles can I expect after doing this?

You used an invalid payment method during your last attempt to create an agreement. Provide a valid payment method and then try creating the agreement, again. For more information, see Payment errors La in the AWS Marketplace Buyer Guide.

I use Terraform for most of my projects My approach is usually to set things up on the console for services i never used before to get acquainted with it, once i have a working configuration i would mimic the same in terraform For services i am familiar already, i would go straight and write terraform code

However i never got a chance to get into either CDK or Cloudformation. Is there any benefits or that is a refundant skill for me given i use Terraform already?

I currently have a site to site vpn tunnel setup between an azure network and aws network. After creating the connection, everything works fine for about an hour and then the aws side drops the connection, but the azure connection stays active and connected. Has anyone else experienced anything like this?

I've got a glue ETL job setup to flatten and relationalize json files and upsert to redshift. Everything works very well. The business requirements were to expect batches of about 20 to 50 files every half hour for a downstream bi tool. Now they've come and said they need to send us 73,000 files as a one time export from their source system that is being sunset. I've never run that many files at once. What's best practice for this without blowing up cost? My current pipeline reads from sqs as the batches come in and glue runs on a schedule. So I'm thinking simplify to just reading directly from the bucket, nothing fancy. Then a separate one time upload to redshift after crawling everything into a glue database? Files are like 2mb each, pretty small. Any recommendations and guesses on cost would be great!

I recently had to switch from hosting my LLaMA 3.1 8B on my machine (using TGI, fp16) to using Bedrock for inference. They are like completely different models. The one on Bedrock doesn't follow instructions at all if the prompt is even relatively long. Is this a common thing? I can't find the info about whether Bedrock host quantized or unquantized model, but the output from it looks similar to a very low quantization.

Hello all, So, i have some questions i couldn't find a straight answer to:

1. In which case is it helpful/necessary to install AWS Load Balancer Controller (https://docs.aws.amazon.com/eks/latest/userguide/lbc-helm.html#lbc-helm-install) ?

2. Isn't it installed already when launching an EKS cluster (creating a service of type LoadBalancer effectively launches a classic LB, so...) ?

3. When deploying a service (kubectl apply service-xyz.yaml) of type LoadBalancer, it creates a classic LB. Is there a way to create an ALB instead?

My understanding is that the above is a solution, but i cannot find an example (I tried creating a service with annotations: service.beta.kubernetes.io/aws-load-balancer-type: "application") but it creates an NLB instead

4. Since deploying a service creates a load balancer, what is the point of creating an ingress? Are they mutually exclusive or can be used together somehow? I can manage routing using an ALB host rules, which seems to be one of the advantages of an ingress

My objective is to understand how vanilla k8s work, and learn about the specifics of EKS as well. My go to was always ECS for deploying containerized workloads, microservices... but i am getting more into Kubernetes after a long breakup :grinning:

I'm looking to provision an SQL database using services like DigitalOcean, Linode, Vultr, or AWS, but there’s a good chance that I might host my Node.js API on Vercel, where I have experience deploying to it.

For security reasons, I want to set up this API to interact with the database, as my application is a small WPF desktop app that will be used by no more than three users from their personal computers.

I have experience creating a Node.js API without any security features, primarily for testing. However, I now need to secure both the API and the database.

I realize that security can be a vast and complex subject, but I'm looking for some baseline practices that will allow me to achieve a reasonable level of security without diving into overwhelming details.

What are some practical steps or recommendations you would suggest for securing the API and the database in this scenario? Thank you!

So quick question, is there a straight-forward programmatic way to trigger some code whenever a user verifies their new email address? I know the custom email sender lambda has a trigger source for the initial part (UpdateUserAttributes), but I'd want to hook into the verification part as well. From my research, the only way to do it is to enable management events in a CloudTrail trail and then add an EventBridge rule with a lambda target, which seems overkill since there's no way to filter for specific events, and I feel like all in all costs would end up being disproportionate.

For a bit more context, I'm trying to update the database email of a user whenever their identity provider one changes, which seems like a pretty normal use case which is why it seems surprising that there isn't an apparent solution for this.

We provided hourly, daily, and monthly database backups to our 700 clients. I have it setup for the backup files to use "hourly-", "daily-", and "monthly-" prefixes to differentiate.

We delete hourly (hourly-) backups every 30 days, daily (daily-) backups every 90 days, and monthly (monthly-) backups every 730 days.

I created S3 Lifecycle Rules (three) for each prefix, in hopes that it would automate the process. I failed to realize until it was too late that when setting the "prefix" for a Lifecycle rule to target literally means the whatever text (e.g., "hourly-") has to be at the front of the key. The reason this is an issue, is the file keys have "directories" nested in them; e.g. "client1/year/month/day/hourly-xxx.sql.gz"

Long story short, the Lifecycle rules will not work for my case. Would using AWS Lamdba to handle this be the best way to go about it? I initially wrote up a bash script with the intention to have run on a cron, on one of my servers, but began reading into Lambdas more, and am intrigued.

There's the "free tier" for it, which sounds extremely reasonable, and I would certainly not exceed the threshold for that tier.

When you use CDK, you don’t have control over the generated CloudFormation, so you won’t know what to change in the console if you need to fix something quickly.

This was said to me today by my manager while we were talking about the various options available for IaC. Our infra is 100% CloudFormation by his choice (not even SAM, just straight CF).

Can someone please tell me I’m not crazy for thinking that this is a wild take? Putting aside the fact that we’re click-op’ing in prod, why would the IaC language affect your ability to change things in the console? It’s not like you’ll have no idea what resources will be created…

But I’m just a junior so what do I know

Have a phone screen interview in a week a and was super anxious, the OA contained some systems design questions and recruiter said it might come up, with technical and LPs.

Anyone can offer advice, how to prepare?

Hey everyone,

I’ve been working with ECS services and tasks, and I’ve noticed that the ENIs (Elastic Network Interfaces) created during task startup are often hard to track down when trying to troubleshoot networking issues or identify which task/service is associated with a specific IP address.

I’m wondering if there’s a way to automatically add tags to these ENIs. Specifically, I’d love to have tags like the ECS service name, task definition, or something similar to help quickly identify which resource created a particular ENI. Or, at least, add a column in ECS tasks so I can see the private IP Address of a service.

Does anyone have any experience with this or know of any solutions? Any tips or insights on making this tagging process easier or automating it would be really helpful for troubleshooting and network mapping.

Thanks in advance!

Hi all,

I'm doing a project called Cloud Resume Challenge to help me learn AWS and develop my skills. I have a single domain name for my resume website that I want to be done entirely in AWS.

The problem I'm running into is that I have sub pages in different folders. Like myname.com/projects/index.html, but I want my links to be clean and only use myname.com/projects/ to load the page.

I've tried a lot of things and it doesn't seem I can get around this limitation. As I understand it, S3 doesn't have the concept of folders and I'd have to explicitly add the index.html to my subfolder link.

For those that have done the Cloud Resume Challenge, is there some reason I should stick with the S3 bucket? Will I be missing out on learning something important if set up an EC2 linux web server instead?