Hi,

is the 3 months free Lightsail trial overall for 750h a month, meaning for Windows and Linux combined. Or can I run 750h of Windows and also 750h of a Linux lightsail a month?

I see knowledgeable devs advocate for DynamoDB but I suspect it would just slow you down until you start pushing the limits of a RDBMS. Amplify's use of DynamoDB baffles me.

DynamoDB demands that you know your access patterns upfront, which you won't. You can migrate data to fit new access patterns but migrations take a long time.

GSIs help but they are eventually consistent so they are unreliable - users do not want to place a deposit then see their balance sit at $0 for a few seconds before bouncing up and down.

Compare this to a RDBMS where you can query anything with strong consistency and easily create an index when you need more speed.

Also, the Scan operation does not return a consistent snapshot, even with strongly consistent reads enabled - another gotcha.

Hello fellow engineers, I am having trouble finding the correct answer to this question.

I have a certain EC2 server with ubuntu 22.04 installed and a java app running on it (linux service). This begun as a small app so I used to t2.medium for it which is using the i386 architecture (did not know at the time). I now need to upgrade this server to a stronger one, and I was thinking t2.xlarge which is using the x86_64 architecture. If I just resize the server (change the type) will the app and everything on the server work regularly or can there be some possible problems there?

What are my options here?

Thanks in advance. :)

.Net
App will run on prem or Azure
The queue receives ~100K messages/day.
Messages are unevenly distributed in time. The amount of messages peaks on certain hours (see image).Processing of a message takes between 200ms and 5s. ~90% finish in 1s.
I need to receive the messages as soon as they appear in the queue and process them. Minimizing delay is commendable.

https://preview.redd.it/ph6opu6vxt4e1.png?width=402&format=png&auto=webp&s=beb7feb6da1b7ae76f8e851b356a3e057daf40b4

My questions:
Should I have a single `SQSConsumer` consuming in a infinite loop and push messages to some in-memory queue? (Where I consumer and fire async processing)
Should I have multiple `SQSConsumer` consuming?
Should I have a single or multiple `AmazonSQSClient`?
Should there be no internal queue and should I run ~20 concurrent (threads) consumers each with its own `SQSConsumer`?
Is it recommended to persist messages on prem?
Should I use long or short pooling?

Please advice :)

I am trying to be PCI DSS compliant by having end to end encryption. I am using ECS Fargate, and was wondering if anyone has been able to do end to end encryption somehow? I think Service Connect may work but I am unsure if I need to configure my containers with nginx etc. Any guidance or general discussion about this would be appreciated!

Aurora DSQL announced y'day in re:Invent 2024 https://aws.amazon.com/blogs/database/introducing-amazon-aurora-dsql/ - some of the very interesting features are:

- Multi Region Active-Active

- Strong Consistency across mulktiple regions

- Serverless

- Low Latency

Is this the true equivalent to DynamoDB NOSQL database but in the SQL world?

We're currently using db.m6g.2xlarge and are considering whether the more expensive db.m7g.2xlarge is worth it or not

Our application is a WhatsApp Marketing + Customer Support SaaS. Main points of isses are at

1. the recent chat list load query should be fast
2. when multiple broadcasts are running, the load does affect the chat load speeds, which isnt an ideal experience
3. if we keep read replicas, the replication lag makes the chat a bit out of sync at times

So first we're ofcourse spending more time on making the architecture + queries more efficient but thought if upgrading the m6 to m7 is worth it or not?

found this one article which recommends it for ec2 for sure: https://www.learnaws.org/2023/02/25/m7g-m6g/

Our free memory most of the time is just 20gb out of 32, so definitely not a memory hungry application.

By the way, I have just switched to db.m7g.2xlarge, so will definitely know soon, but what has been your experience if you're also using rds? m5 vs m6/7 is a no brainer because of gravitron, but not sure about this one

I'm building an App (non profit) and for now I use Aws Amplify, S3 and DynamDB. Honestyl I'm not really happy with that. I feel like it's way to complicated and it's very slow. I don't know why but for like 30 items in my DynamoDB it takes seconds to get the items. This is no acceptible! Because what if i have more then thousand of items. Would it takes multiple seconds or worse minutes to get the request fulfilled.

So my Question is. IS this a generel problem with AWS? And what are good or better alternatives? I heard about Supbase. And they using postgressql and it looks like i can connect prisma too. I like to work with prisma and it's fast too.

But how is the pricing? Is it cheaper than aws ? What are the key points i have to take a look?

Or is it better or possible to host on cloudflare your own backend? I mean i need just an Api Backend maybe with Nestjs. And of course an Authentication. Maybe OAuth2.

But i could not find any resource how to do this. Is it possible to host a nestjs app on cloudflare. On the pricing i saw 100k requests/day on the free plan this would be really enough on the first step.

Do they really mean that as I understand, an api request to my backend.

Or what is meant by 100k/request per day.

I hope someone can give me some tipps.

So I have been using AWS services for almost a year now and have promptly paid every month's bills. This time, I am running low on the bucks.

The earliest I can pay is, the 5th of next month even if it means paying the aggregate of 2 months.

Will my account be suspended or my instances be shut down if I delay the payment?

UPDATE: I contacted support and I've gotten approval for an extension till the 10th of Jan as a one time exception. If anyone is curious, the amount is small, about $26.

Hi all,

I have an ingestion pipeline with glue and s3 combination. S3 buckets will be used for Raw layer, Publish layer and Functional layer.

Glue job between Raw and Publish will validate the schema.

I have to do this for 1000 + files which will run on daily basis.

I am looking at creating a generic job to which we will pass file name, it's schema as a parameter. The job will be same and only the parameters will change. It will save a lot of development time.

I would be grateful if you can provide your input on this.

P.S new to aws

If you need a ticket hmu - looking for 20% or less of cost

I was practicing with a s3 static website, deleted everything and started from scratch again, now this time my ACM is taking too long to validate, could it be that i used the same names for the bucket, cloudfront, and route53?

The following policy blocks write access to all the buckets. How to grant read access to one bucket and full access to all other buckets?

####

{

"Version": "2012-10-17",

"Statement": [

{

"Sid": "FullAccessToAllResources",

"Effect": "Allow",

"Action": "*",

"Resource": "*"

},

{

"Sid": "DenyWriteAccessToSpecificBucket",

"Effect": "Deny",

"Action": [

"s3:PutObject",

"s3:DeleteObject",

"s3:PutObjectAcl",

"s3:DeleteObjectAcl",

"s3:PutBucketPolicy",

"s3:DeleteBucketPolicy"

],

"Resource": [

"arn:aws:s3:::my-read-only-bucket",

"arn:aws:s3:::my-read-only-bucket/*"

]

},

{

"Sid": "AllowReadAccessToSpecificBucket",

"Effect": "Allow",

"Action": [

"s3:GetObject",

"s3:ListBucket"

],

"Resource": [

"arn:aws:s3:::my-read-only-bucket",

"arn:aws:s3:::my-read-only-bucket/*"

]

}

]

}

I’ve got some layers that are over the 250mb size limit for lambda layers. Has anyone got any recommendations for what I can do to get around this?

I am trying to design the architecture to a website that allows users to upload pdfs and they can choose a variety of options for how they would like the files manipulated.

I'm using cognito for user management and amplify to host the site. The next and biggest hurdle is the file management. It is my understanding the most common approach is to use a single S3 bucket and giving users presigned urls to upload to it. The question I run into is where in that process do you assign the information to a table to ensure users cant see each others files? If a user logs in what resource would you use to quickly supply them with their files and no one elses? Should I be using cloudfront instead? Should i consider an architecture where every user gets their own subdomain in an S3?

Hey folks, we jotted down some notes from the AWS re:Invent 2024 opening keynote, led by Matt Garman in his debut as AWS CEO. If you missed it, here’s a quick rundown of the big announcements and features coming in 2025:

• Compute

1. Graviton4: More powerful, energy-efficient, and cost-effective than ever. Graviton4 delivers 30% more compute per core and 3x the memory compared to Graviton3. It’s already helping big players like Pinterest reduce compute costs by 47% and carbon emissions by 62%.
2. Trainium2 Instances: Now GA! Boasting 30–40% better price-performance than current GPU instances, they’re purpose-built for demanding AI workloads.
3. Trainium2 Ultra Servers: For those training ultra-large models, these babies combine 64 Trainium2 chips for 83 petaflops of power in a single node. Anthropic’s Project Rainier is leveraging these for a 5x boost in compute compared to its previous setup.
4. Trainium3 Announcement: Coming next year, this next-gen chip promises 2x the performance of Trainium2 while being 40% more efficient.

• Storage

1. S3 Table Buckets: Optimized for Iceberg tables, these offer 3x better query performance and 10x higher transactions per second compared to general-purpose S3 buckets. Perfect for data lakes and analytics.
2. S3 Metadata: Automatically generates and updates object metadata, making it easier than ever to find and query your data in real-time.
3. Cost Optimization: Tools like S3 Intelligent-Tiering have saved customers over $4B by automatically shifting data to cost-efficient tiers.

• Databases

1. Aurora D-Seq: A distributed SQL database offering low-latency global transactions, 5-nines availability, and serverless scalability. It’s 4x faster than Google Spanner in multi-region setups.
2. Multi-Region Strong Consistency for DynamoDB: Now you can run DynamoDB global tables with multi-region strong consistency while maintaining low latency.

• Generative AI & Bedrock

1. Bedrock Guardrails: Simplifies adding responsible AI checks and safety boundaries to generative AI applications.
2. Automated Reasoning Checks: Ensures factual accuracy by verifying model outputs mathematically—critical for high-stakes use cases like insurance claims.
3. Bedrock Agents with Multi-Agent Collaboration: This new feature allows agents to work together on complex workflows, sharing insights and coordinating tasks seamlessly.
4. Supervisor Agents manage dozens (or hundreds!) of task-specific agents, deciding if tasks run sequentially or in parallel and resolving conflicts. For example: A global coffee chain analyzing new store locations. One agent analyzes economic factors, another local market dynamics, and a third financial projections. The supervisor agent ties everything together, ensuring optimal collaboration.

Edit:

• Data Analytics

1. S3 Tables: Optimized for Analytics Workloads
AWS unveiled S3 Tables, a new bucket type designed to revolutionize data analytics on Apache Iceberg, building on the success of Parquet.

• Why It Matters:
  • Apache Iceberg is a leading format for large-scale analytics, but managing it traditionally requires manual maintenance and complex workflows.
  • S3 Tables automate optimization tasks like data compaction and snapshot cleanup, eliminating the need for customers to schedule Spark jobs.
  • The new buckets offer 10x performance improvements for Iceberg-based analytics workloads by pre-partitioning buckets and streamlining operations.
• Features:
  • Iceberg catalog integration with first-class table resources.
  • Enhanced access control and security at the table level.
  • REST endpoint for seamless query integrations.
• Performance Gains:
  • Dramatic reduction in the overhead associated with maintaining large Iceberg tables.
  • An estimated 15 million requests per second for Parquet files highlights the demand for these enhancements.

2. S3 Metadata: Accelerating Data Discovery
The S3 Metadata feature addresses the pain point of finding and understanding data stored in S3 buckets at scale.

• How It Works:
  • Automatically indexes metadata from S3 objects, storing it in an Iceberg table for fast querying.
  • Enables users to run SQL-like queries to locate objects based on parameters like file type, size, or creation date.
  • Metadata updates occur in near real-time, keeping queries accurate and up-to-date.
• Use Case: Instead of manually building metadata layers, customers can leverage this feature to streamline analytics workflows.
• Integration: Works seamlessly with Amazon Athena and other Iceberg-compatible tools.
• Amazon Sage Maker

1. SageMaker Unified Studio:
   • A single development environment for data discovery and cross-functional workflows in AI and analytics.
   • Integrates tools from Amazon EMR, AWS Glue, Amazon Redshift, Amazon Bedrock, and SageMaker Studio.
2. SageMaker Lakehouse:
   • An open data architecture that unifies data from Amazon S3 data lakes, Amazon Redshift warehouses, and third-party sources.
   • Supports Apache Iceberg-compatible tools for flexible data access and queries.
3. SageMaker Data and AI Governance:
   • Includes SageMaker Catalog (built on Amazon DataZone) for secure data discovery, collaboration, and governance.
   • Streamlines compliance and ensures secure handling of data and AI workflows.

• Nova:

AWS unveiled Nova, a new family of multimodal generative AI models designed for diverse applications in text, image, and video generation. Here's what's new:

1. Nova Text-Generating Models

• Four Models:
  • Micro: Text-only, low latency, fast response.
  • Lite: Handles text, images, and video; reasonably quick.
  • Pro: Balances speed, accuracy, and cost for multi-modal tasks.
  • Premier: Most advanced; ideal for complex workloads and custom model training.
• Capabilities:
  • Context windows of up to 300,000 tokens (225,000 words); expanding to 2 million tokens in early 2025.
  • Fine-tunable on AWS Bedrock for enterprise-specific needs.
• Use Cases:
  • Summarizing documents, analyzing charts, and generating insights across text, image, and video.

2. Generative Media Models

• Nova Canvas:
  • Creates and edits images using text prompts.
  • Offers control over styles, color schemes, and layouts.
• Nova Reel:
  • Generates six-second videos from prompts or reference images, with customizable camera motions like pans and 360° rotations.
  • A two-minute video generation feature is coming soon.

3. Responsible AI and Safeguards

• Built-in watermarking, content moderation, and misinformation controls to ensure safe and ethical usage.
• Indemnification policy to protect customers from copyright claims over model outputs.

4. Upcoming Features

• Speech-to-Speech Model (Q1 2025):
  • Transforms speech with natural human-like voice outputs.
  • Interprets verbal and nonverbal cues like tone and cadence.
• Any-to-Any Model (Mid-2025):
  • Processes text, speech, images, or video inputs and generates outputs in any of these formats.
  • Applications include translation, content editing, and AI assistants.

That’s the big stuff from the keynote, but what did you think?

Hi everyone,

I'm looking for advice on selecting the right service or combination of services for my specific use case. I need to process new files stored in S3, and I'm aiming to handle them in batches with near-real-time processing.

The files should be grouped based on a particular property found in the S3 event path and timestamp (part of the file name). While there usually aren't many files to process, there are occasions when the files may be larger (up to tens of MB). I'm confident that AWS Lambda can manage the whole processing of these files, even when grouped into batches.

Typically, the files are uploaded within a few minutes, but sometimes the upload process can take longer, and unfortunately, I can't modify this, at the same time I can't get a piece of information that files upload for specific timestamps have ended. Each file is timestamped to the nearest minute.

In essence, I receive S3 event notifications and want to group the events by a path property and their timestamp. Once the events for a given timestamp have stopped coming in (let's say for a minute), I want to send this batch for processing. I would say that overall there will be hundreds of such batches with tens of files for each minute.

I'd appreciate any recommendations or insights on how to best accomplish this. Thanks in advance!

Cloudfront has a compute utilization metric. 71-100 means that your function may suffer from throttling.

How does this behave in practice? Will the execution of the viewer_request function be ignored and the request proceed or will Cloudfront throw an error? If an error is thrown, how do you fallback to it?

hey all,

is anybody out there in the real world who was able to set up Chatbot with Slack?

maybe it's a #skillissue on my end but two times I tried, two times I failed.

I hit a roadblock either when EventBridge "transforms" the message from CloudWatch or SNS fails with some "unsupported event"-type of error.

is there a github repo I can maybe look into? or a blog post from anybody?

thank you!

Hello everyone, I am currently learning to use AWS through a project and I am having trouble getting my app to talk with my postgres DB. So here's the setup:

• The app is a flask/bootstrap app which runs fine locally (both with flask and Docker)
• The app is pushed via Git actions, git holds the secrets for Postgres, etc, the workflow creates a task definition along the way.

https://preview.redd.it/fd6hr25tmn4e1.png?width=512&format=png&auto=webp&s=cb727dda1b4736b11121a617e60caf3985ec1cda

• In AWS, the app is in an ECR container, there's an ECS cluster, EC2 instance... Everything is working quite fine except when the app submits or try to query data from RDS.
• Also my IAM users has a permission "AmazonRDSFullAccess"
• The database credentials management is "self managed" with a username & password (database authentification is set to password authentification)

My postgres db on RDS works well via pgAdmin

I was suspecting security groups but I can't figure out or find a way to debug.

Speaking of SG:

Any help would be greatly appreciated. Thanks!

We have an app which uses DB triggers to update certain rows (i.e, updating a user table might then trigger an insert into another table).

We use Elasticache Serverless (valkey) to cache db queries, but where we're getting stuck is when a trigger updates a row, we need to invalidate the cache for that row, AND the cache for other rows which the trigger might have created/updated. The application itself has no knowledge of what the trigger may have done.

How would you design a caching strategy to handle this? MySQL triggers can't call lambda functions, or talk to Elasticache directly. So it seems like you'd need a service to monitor all writes to the DB, then somehow invalidate the cache...?

(It might just be, don't use DB triggers you idiot. But I'm curious if there's a way to make this work.)

Is it possible to start a cloud service business by specializing in AWS? I’m wondering if deep knowledge of AWS could allow me to offer services to companies already using it or help those not yet on the cloud with implementation and optimization. I’d love to hear thoughts or experiences from others! Ps ( im at the beginning of my journey im studying to get my cloud practitioner cert to start 😊)

Hi,

I live in Europe so it's difficult to watch the talks in real-time because of the time difference. I checked out the AWS Events YouTube channel but only 2-3 talks are available.

I'm interested in a couple of the upcoming sessions, but the timing makes it impossible to watch them live. Is there a place where I can watch the recordings afterward,m or are they not available?

Thanks!

All the tutorials and guides when googling this issue go this way, 'how to Geoblock using ACLs/ WAF / other AWS things' But I need to remove it so I can gain access from worldwide ideally, Assumed it should just be this way by default and the guides infer this too.

I setup an EC2 instance and in the Security group O opened a port range and now they are open and working in my country (AU) but not from NZ, browser just shows network/site unreachable.

Can someone guide me to where I need to go to remove any blocking that may be in place or provide an explanation as to why it's operating that way.

This is one EC2 instance launched with next to no config other than the elastic IP applied, I can remote into the instance no issue. but just not other countries.

Where can I find information about the options for networking receptions tonight?

Hello, AWS community,

I'm looking for some advice and insights on how to transition into a role as an AWS Security Architect.

A bit about me:

• I'm in my mid-40s and have 6 years of experience as a backend developer, a career I transitioned to as a self-learner.
• Over the past two years, I've gained hands-on experience with AWS, including:
  • Setting up and managing EC2 instances, S3 buckets, RDS databases, and SES.
  • Migrating resources between regions.
  • Implementing CodeDeploy for CI/CD pipelines.
• On the development side, I've built APIs using TypeScript and Python (FastAPI).

I'm now looking to grow into a cloud architect role, specifically focusing on security architecture. My challenge is understanding the most effective roadmap to get there.

I’d love to hear your recommendations on:

1. Certifications that would be most relevant and impactful (e.g., Solutions Architect Professional, Security Specialty, etc.).
2. Skills and tools I should focus on to bridge the gap between my current role and the security architect position.
3. Any resources, courses, or practical labs you’d recommend for someone on this journey.

I deeply appreciate your help and any guidance you can offer.

Thank you in advance!