/r/aws

Photograph via snooOG

News, articles and tools covering Amazon Web Services (AWS), including S3, EC2, SQS, RDS, DynamoDB, IAM, CloudFormation, AWS-CDK, Route 53, CloudFront, Lambda, VPC, Cloudwatch, Glacier and more.

News, articles and tools covering Amazon Web Services (AWS), including S3, EC2, SQS, RDS, DynamoDB, IAM, CloudFormation, AWS-CDK, Route 53, CloudFront, Lambda, VPC, Cloudwatch, Glacier and more.

Note: ensure to redact or obfuscate all confidential or identifying information (eg. public IP addresses or hostnames, account numbers, email addresses) before posting!

Smokey says: avoid streaming video to fight climate change! [see more tips]

If you're posting a technical query, please include the following details, so that we can help you more efficiently:

  • an outline of your environment
  • a description of the problem
  • things you've tried already
  • output that was displayed (if any)

Resources:

Sort posts by flair:

Other subreddits you may like:

Does this sidebar need an addition or correction? Tell us here

/r/aws

311,858 Subscribers

1

Does anyone know any useful website like "IAM Trust Relationship" generator ?

Does anyone know any useful website like "IAM Trust Relationship" generator ? I found this one (https://www.awsiamactions.io/generator) which is really cool in terms of policy generator. I'd like something similar to create trusts.

0 Comments
2024/11/09
20:19 UTC

1

My website on AWS keeps saying it's not secure

First off, let me say that I'm not really versed with using AWS. I have a static website for showing my work. No interactivity to it. A friend set this up on AWS years ago and all I do is upload the site i created when it needs updating.

So, my question is, how can I make it that when someone visits the site that they don't get the site not secured message? I see there are a few threads on this, but they are a few years old. So, i'd like to see if I can get the most up to date information.

Thank you.

5 Comments
2024/11/09
20:09 UTC

3

Creating Cognito User Pool: benefits/risks of social sign in?

I'm trying to figure out if it's a good idea to allow the users of the app I'm building for a client to sign in with social accounts

Is this a good idea? Are there benefits to this for me or my client? Are there risks that I'm not noticing by allowing this?

3 Comments
2024/11/09
20:04 UTC

1

API Gateway and Lambda?

I'm planning on building an iOS mobile app and was looking at using API Gateway, Lambda and RDS (amongst other services) as the backend.

I'm curious if it is a good idea using these services from the start? I've heard positive and negative things about serverless backend and I'm curious what people really feel about it.

What is considered to be best practice for mobile backends? What would you use?

3 Comments
2024/11/09
19:16 UTC

1

Are API Gateway deployments eventually consistent?

I updated the authentication mode of an API gateway resource to NONE. I deployed the API Gateway. I issued a series of requests through curl immediately.

Initially, the requests were failing with 403 (as if requests were routed to the version with auth enabled). Later I could see the expected response intermittently along with 403 errors. After some time all the 403s disappeared.

In my experience it took around 20s to notice consistent results.

My set-up: API gateway is configured to trigger lambda function. All resources are in us-east-1 with no cross account or cross region business at all. Before changing security to NONE for the target resource, I was using custom lambda authorizer. Only the security for the resource is modified, the resource API lambda and custom authorizer lambda (if it matters) was unchanged.

So, do I have to factor in delays when deploying API gateway?

3 Comments
2024/11/09
17:45 UTC

0

breaking into finding aws work

i'm struggling , i know the market isn't great. i have a the solutions architect cert, a dept cert from Santa monica college (and almost done with an AA...i have a bfa in another field) i've been applying to internships, i'm older so i think thats why i'm being passed. i'm coming from an edit/animation bg, so always have been a bit on the techy side.

Any suggestions? I'm constantly emailing, applying etc...i know the market's not great....i'm based in LA county

9 Comments
2024/11/09
17:32 UTC

4

SQS message body cut off at ampersand

Hi all, trying to debug an issue with an API Gateway -> SQS -> Lambda setup.

I'm running an app with a Supabase backend. I have a DB webhook set up that calls my API Gateway endpoint on INSERT with the new record body.

These records are for recipes, and contain properties id and recipe_src (among others, those are the important ones).

This morning, I noticed this message in my DLQ (some attributes removed):

{
   "Records":[
      {
         "body":"{\"type\": \"INSERT\", \"table\": \"recipes\", \"record\": {\"id\": 251, \"fat\": null, \"carbs\": null, \"title\": null, \"yields\": null, \"og_data\": null, \"protein\": null, \"user_id\": \"<redacted>\", \"calories\": null, \"favorite\": false, \"created_at\": \"2024-11-09T14:10:16.813323 00:00\", \"deleted_at\": null, \"image_path\": null, \"recipe_src\": \"https://recipesbyclare.com/recipe:cheesy-garlic-chicken-wraps?utm_source=Krm",
      }
   ]
}

As you can see, the body of this record isn't even valid JSON... it was cut off halfway through the recipe_src URL.

The actual URL stored in my DB is this:

https://recipesbyclare.com/recipe:cheesy-garlic-chicken-wraps?utm_source=Krm&fbclid=IwZXh0bgNhZW0CMTEAAR3ObUm0qlR6Xo6UUv3uafnZrNsPTH514trCjgd7loRbUjS5YbGxRuFXDWA_aem_tJ9uZXyNnkyITyj39e0xzg

At first I thought it was some random size-based truncation, but comparing the message body to the actual URL, it was cut off at the first & character.

Still not 100% sure if this is happening in my AWS setup or if I just got a bad message from the Supabase webhook -- but assuming I got a valid event from Supabase, any ideas on why the message body was cut off at the &?

Thanks!

6 Comments
2024/11/09
17:09 UTC

6

App Runner underated ?

When there are technical discussion about diagrams, no one think of AppRunner service. I think it is quite easy to run/use. Is it due to cost ?

5 Comments
2024/11/09
16:14 UTC

0

Getting connection timed out on Mobaxterm while accessing ec2 instance

I'm having trouble accessing my EC2 instance using MobaXterm. Every time I try to SSH into it, I get a "Connection timed out" error. Here’s what I’ve done so far:

Instance Type: [t2.large]

OS: [Ubuntu]

Security Group:

Port 22 (SSH) is open to my IP (also tried 0.0.0.0/0 for testing).

Key Permissions: Set to chmod 400 on the .pem file.

2 Comments
2024/11/09
10:09 UTC

0

Need help understanding my bill and cost management for free tier resources that are charging me.

I set up a React/Node/MySQL website at the end of October. I serve the react front end from S3 using a cloudfront distribution.

The Node app is on a single EC2 instance. It's a Free Tier t2.micro running Ubuntu. I've only installed the Node app and Caddy as a reverse proxy tool.

The RDS uses MySQL Community on a Free tier 'db.t4g.micro' instance with 20GB of storage. At the end of october I inserted about 300MB of data to it.

I've set up a Budget for $25/month, moreso as a safeguard (I never thought I'd actually see it hit $10). I just received an email that I'm on pace to hit $27 (chiefly because of RDS and EC2, but a few other expected resources like route53/cloud dist)

I currently have no traffic to my website. I am barely testing the site myself, visiting it once every few days. The workload when I do is minimal. It's a simple CRUD app serving simple "book" resources. I have no test suites that run, and no custom health checks (not sure if AWS does their own that would cause charges).

Almost all RDS metrics sit idle at zero. The only metric I see that piques my concern is that CPUCreditUsage hovers at 0.3 at all times. I have no idea why. At the moment the Cost Management tool says that RDS has charged me $4 and is on pace for $13/month.

I realize this isn't a crazy amount of money, but when you're expecting free and you end up getting a bill for $27, it's a bit of an eye opener! And maybe I'm just new to AWS and missing where to find the info, but I can't see anywhere that breaks down the cost of a resource's usage (e.g. by credit usage, storage, in vs outflux, etc.)

screenshots of RDS graphs

5 Comments
2024/11/09
13:11 UTC

0

How to get computing power on EC2?

Hi, n00b here! Just on the free tier of AWS/EC2. I don’t have much experience or knowledge on anything tech or IT but I have some code I want to run for heavy data analysis. ChatGPT has been helping me with everything hahaha

I have messed around and spun up a t2 instance running Ubuntu. Used my key pair and terminal to SSH into the server, SCP my .py and .csv on. Run the code, got the output file etc

But it was super slow, my laptop was faster. No surprises.

I also tried a Spot Price Instance but wasn’t able to get any to fill for Ubuntu and only around 30 vCPUs when I selected a Windows OS but I had (skill) issues getting anything done in Windows, i.e. installing Python, transferring files

So 2 questions: is EC2 the best way to achieve what I want to do? And what do I need to do to get more processing power/exit the free tier? Do I have to contact sales to get a dedicated instance with more CPUs?

10 Comments
2024/11/09
08:55 UTC

1

[Question] AWS Athena and Glue Python queries

Hey Redditors, I need your help

I am attempting to build a Python Lambda function to pull data from multiple Athena databases using AWS Wrangler Python library.

wr.athena.read_sql_query('across databases sql query', 'one of databases name')

This call is not throwing all kind of permission errors:

  1. It throws an exception complaining that the table exist on a different AWS account under the same organization (Is that possible?)
  2. Or it complains that it doesn't have permissions to the output s3 bucket (which can be found in the settings tab of Athena). Not sure how that is possible?

Any comment could help here.

0 Comments
2024/11/09
04:52 UTC

0

Access S3 uploaded file by authorized user only

I'm a noobie in AWS S3 and I have used it in one or two projects, and those uploaded files are public and can be accessed by anyone(if they got a specific URL for the file). So, My question is, Is there any way to access the file for authorized users only(authorized users means the authorized users from my web app not based on was users)

4 Comments
2024/11/09
08:54 UTC

0

Cloufront cors error

Facing cors error only for json and html files, css js files load properly. What could be the reason? The earlier distributions we created work fine, the settings are same as previous ones but now the same settings aren't working. Anything changed recently?

Also we are using a custom origin.

5 Comments
2024/11/09
05:19 UTC

0

How do I get the access of AWS academy for solution architect associate?

I m currently studying the solution architect associate course offered by government program. Our teacher teach us from AWS academy, when we asked for the access he said it is a copyright material and he is not allowed to give the access.but prior to solution architect associate he taught us cloud foundation course he gave us the access of AWS academy.

0 Comments
2024/11/09
05:12 UTC

1

Subject: Seeking Resources for Real-World Architecture Artifacts Examples

As a senior solution architect working on complex projects, I’m looking for real-world examples of architecture artifacts. Specifically, I’d like guidance on typical artifacts I should produce, like functional/non-functional requirements, customer journey mappings, current and target state documentation, and preparing for architecture review boards. I’d prefer practical examples or resources over frameworks to see how these are applied in real settings. Any top websites or resources you’d recommend for reference? Thanks!

0 Comments
2024/11/08
17:56 UTC

1

How do you guys use the tutorials in AWS documentation?

Do you typically do all the tasks, or do you only do the specific things you need? I'm a beginner with AWS and sometimes feel like I get more out of targeting specific tasks, but I'm wondering if I’m missing out by not doing the whole thing. For example, I am learning to launch an instance but the first 2 tasks of this tutorial are more conceptual. I'm more inclined to start at creating a key pair. So I am just wondering how others do it.

0 Comments
2024/11/08
15:05 UTC

5

Is lambda the right approach here?

I have 40+ eventbridge schedules (cron jobs). They are just api calls to my application. Right now, i have all of these schedules pointing to a single lambda target that executes the api call through a private load balancer. That's a simple explanation of my current workflow.

Some of my jobs can take a long time, and given that lambda only has maximum of 15 minute execution time, i wonder if I should be using some other workflow. Right now i'm trying to understand if I can use some fire-and-forget type of pattern on the lambda that would execute the api call and exit immediately without waiting for the call to finish. Is something like this possible?

Using python for the lambda btw.

19 Comments
2024/11/09
02:04 UTC

0

Loop - low performance on 1 interview

Hey Everyone,

I just did my loop interview. I am confident that I showed strong performance on HR and BR manager interviews. However, at some point I got tired and I underperformed during one of the interviews. I have been asked “how do you do….” , and I thought it was a casual chat, instead of a story i had to tell 🤦🏻‍♂️ .. it went downhill from there…

Am I doomed or do i still have a chance? Note: I have the deep domain expertise for the position they are looking for.

Thx! I am overthinking about this a lot!

6 Comments
2024/11/09
00:50 UTC

1

AWS RDS Proxy and Nodejs ORM

Hello. I have an AWS RDS Proxy on top of my RDS instance (PostgreSQL), due to many microservices connecting this DB. I am looking for an ORM that can leverage RDS Proxy's advantages and not causes connection pinnings. I've looked at Prisma, Typeorm, and more, and they all use prepared statements which causes connection pinning.

Do you know of any ORM that can be used so the RDS Proxy's advantages take place.

I know prepared statements is used for security and performance, but is there any ORM that is still protected on the one hand and on the other hand does not use prepares a statement(or let you configure it yourself), like node-pg let you execute parametrized queries without prepared statement. Thanks :)

0 Comments
2024/11/08
19:54 UTC

1

App runner with flask app "gun corn": executable file not found in $PATH

I'm trying to run a flask app on app runner installing from a github report and I can't find anything about anyone else having this error.

I'm definitely installing gunicorn from my requirements.txt, which seems to be the issue for basically every response on Google.

Start command is gunicorn app:app

0 Comments
2024/11/08
19:00 UTC

0

What’s the average monthly salary for a certified AWS Cloud Engineer in the United States?

Hello everyone,

I’m trying to get an idea of the average monthly salary for an AWS-certified Cloud Engineer working in the United States. I’m based in Latin America, where many U.S. companies hire Cloud Engineers. I’m curious to understand the difference in cost for these companies between hiring locally in the U.S. versus in Latin America. If anyone could share salary insights or experiences, it would be greatly appreciated! Thanks!

12 Comments
2024/11/08
23:37 UTC

1

User audit logging

We have a web application based on lambda, we expose the endpoint by API Gateway and then by CloudFront, is there a semi automatic way to save every request into something like an S3? we then would like to query the result but that part is already figured out. I would like to do that without modifying my lambdas.

The most elegant solution feels like to have a lambda edge attached to CloudFront that then pushes the requests to Kinesis and then they are saved into S3. But I would like to do the same thing at the lowest possibile cost without the need to complicating to much the architecture.

10 Comments
2024/11/08
23:03 UTC

2

Need Advice on Optimizing a High-Concurrency Voting System with AWS Lambda and DynamoDB

I’m working on a voting system using AWS Serverless services to handle a large volume of voting requests for a competition. I’d love some input from anyone who’s tackled similar setups or has insights into best practices for high-concurrency environments.

My Stack and Setup

Here’s what I’m using:

  • API Gateway: Manages HTTP requests and handles authentication.
  • AWS Lambda: Processes each vote and makes a transaction request to DynamoDB.
  • DynamoDB: Three main tables are used to track:
  • User votes (with limits per user)
  • Votes by IP address (with limits per IP)
  • Votes per candidate

Each transaction includes conditional checks to ensure no user or IP exceeds the vote limits.

The Challenge

During traffic spikes, I’m encountering “Transaction is ongoing for the item” and “ThrottlingError” errors, likely due to high concurrency on certain DynamoDB items. This causes some votes to fail, impacting the experience during peak voting times.

Current Configuration and Goal

  • Retry Mode: Currently set to adaptive retry mode with 4 retry attempts to manage concurrency.
  • Goal: Ensure that the system can handle 100,000 users voting simultaneously with minimal transaction errors and no cross-tenant impacts.

Looking for Advice On:

  1. Optimizing DynamoDB Transactions: Any tips on optimizing transactions for this setup to handle high concurrency more effectively?
  2. Retry Mode Configuration: I’m considering switching from adaptive to standard mode since standard is generally recommended for multi-tenant applications. Would this help improve stability in my use case?
  3. Best Practices: Any other best practices for high-concurrency, multi-tenant voting systems on DynamoDB? I’m especially interested in how others handle high-throughput voting systems effectively.

Additional Info

I’d prefer to avoid adding SQS to keep the architecture simple and reduce additional complexity. My ideal setup would avoid introducing too many moving parts. With the current setup, is it feasible to achieve high concurrency, or is there a critical gap in my approach?

Additionally, it’s not crucial for the response from Lambda to be immediate via API Gateway, so if the vote processing can be delayed slightly to better handle concurrency, that would be fine.

Any advice, tips, or experience you can share would be hugely appreciated. Thank you!

8 Comments
2024/11/08
22:52 UTC

4

Opensearch cluster KNN Vector scalability

Hello folks.

I am currently moving some old indexes from outdated clusters to a new Opensearch cluster. We have currently "normal" indexes with some searchable core data, as well as one index with KNN vectors plugin.

While planning this migration one colleague suggested that we keep the KNN index in a separate cluster by itself, and add all other normal indices to a second cluster.

The idea behind this idea is that we would be able to buy AWS dedicated instances for the normal indices and scale the node count up if we ever needed it.

And the why to keep the knn index separate is because, in theory, the scalability of the index with this plugin is not throught increasing node counts, but instead increasing the node sizes/memory (which would not work if we have dedicated instance for this cluster). So this cluster would be more flexible and we would not buy dedicated instances for it.

Now I would like to confirm this theory really. Do you agree with this approach? I would like to have a proper piece of documentation stating that but I didn't find any.

Would also be interested to hear any similar experiences you might have.

1 Comment
2024/11/08
22:33 UTC

10

Bedrock getting started

I was following the getting started course that AWS offer for bedrock and I'm upto the point of where I need to configure a model so I can use them. No matter what I do I can't opt in to any of the models.

I've tried messing around with IAM roles and adding the bedrock policies to my root. I've followed a couple of guides on YouTube and there's also work but mine don't, I'm still getting the same issue.

Everytime I load the bedrock page I just get an error which is: provided model identifier is invalid.

I've tried googling that, it just tells me about the API but I'm using the console. I'm not sure if I just picked a bad time to try and mess around with this as I've seen some other posts about quotas which maybe is effecting this?

9 Comments
2024/11/08
21:33 UTC

1

Aws databricks help

Hi fellow engineers, how to get the jobs data of all workspaces and make a pattern out of it using a dashboard maybe a dashboard from databricks itself. Any help would be greatly appreciated.

1 Comment
2024/11/08
18:04 UTC

1

Aws Connect/Lex

How can I prevent the Get Customer Input block in Amazon Connect from reading 'blank' on a loop when using a Set block to pass the initial input

0 Comments
2024/11/08
15:29 UTC

1

Azure Personal Workspace Entra SAML Question

I am working with our AWS team to setup SAML Auth from Entra to AWS for workspaces. The last step of the setup guide https://docs.aws.amazon.com/workspaces/latest/adminguide/setting-up-saml.html#enable-integration-saml has you enabled SAML for your directory, and it states to check the box Allow clients that do not support SAML 2.0 to login. but this specifies its for older clients. If we enable SAML authentication will this force all users to utilize SAML? We just want to test our configuration at this point without effecting users, or is the way to do this with a new workspace directory and enable it on the new directory?

2 Comments
2024/11/08
20:02 UTC

Back To Top