/r/AppEngine

Photograph via snooOG

Subreddit for Google App Engine users & developers worldwide since 2008.

News and articles about Google App Engine.

Please don't post questions here - post them in the official groups, or on Stack Overflow.

Related:

  • /r/bigquery - Google BigQuery, analyzing terabytes in seconds
  • /r/dataflow - Google Cloud Dataflow, the evolution of MapReduce

/r/AppEngine

3,236 Subscribers

1

GAE Go - how to make html templates work?

So I've been working with Google App Engine Go for years, but something changed in the last year or two and I can't figure it out. I'm not able to access my html templates when I try deploying new code.

So what used to work was code like this:

var MainTemplate *template.Template

func init() {
    http.HandleFunc("/", hello)
    MainTemplate, _ = template.ParseFiles("html/main.html")
}

func hello(w http.ResponseWriter, r *http.Request) {
    MainTemplate.Execute(w, nil)
}

I'd simply parse the templates in init() and then Execute them in the various functions. Everything worked like a charm both locally and on GAE. But after coming back to my project after like a year or two, suddenly that doesn't work. It runs correctly locally, but not when it's served online. I get a runtime error: invalid memory address or nil pointer dereference since my MainTemplate is nil. I even tried parsing the template right before using, but that gave me an open html/main.html: no such file or directory error.

I tried re-visiting my app.yaml, but it looks correct to me:

runtime:     go122
app_engine_apis:  true

handlers:
- url: /static
  static_dir: static
- url: /html
  static_dir: html
- url: /res
  static_dir: res
- url: /.*
  script: auto
  secure: always
  login: optional

After digging for like a week I did stumble upon some application_readable parameter, but that seems to not be needed: This field is not configurable with runtime [go122] since static files are always readable by the application. It can safely be removed..

I tried posting the question to StackOverflow, but no answers so far - https://stackoverflow.com/questions/79091929/how-to-upload-and-serve-an-html-template-in-google-app-engine-go . Here is my test repo - https://github.com/ThePiachu/TestGoPrivate/tree/main .

What am I doing wrong? EDIT:

Okay, I solved it. The problem was me conforming to the old old ways of doing GAE - having the app folder as a subfolder of my project. It USED to be the root of the program, so doing template.ParseFiles("html/main.html") worked. But now GAE handles the top project folder as the root, so I need to do template.ParseFiles("app/html/main.html") and also make app/html my static dir...

1 Comment
2024/10/19
20:37 UTC

2

Where can I host a Telegram bot for free?

Hey everyone! I wrote a simple forwarding bot that sends new posts from a selected Telegram channel to my Telegram channel. Now I need to deploy it on some free hosting service.

I first tried PythonAnywhere, but it turns out that free accounts have limited internet access. They can only make HTTP/HTTPS requests to sites from an approved whitelist of domains. Connections using arbitrary protocols and ports (like to Telegram's IP addresses and ports) are not allowed on free accounts.
I then tried Heroku, but it looks like the free tier is no longer available.

Where can I deploy my bot for free?

1 Comment
2024/09/27
15:40 UTC

0

Google App Engine Incident

Hello, everybody! For my graduation, I need to do a deep analysis of that cloud environment, Do you happen to know of any data leak incidents or any incidents at all? And what did Google do as a solution?

3 Comments
2024/09/22
13:41 UTC

1

App Engine Java App stopped working - ideas?

I had a java web app written for me maybe 10 years ago and it has been sitting and running just fine in app engine. Did not do a thing to it in all those years.
Just recently it stopped working. Ideas?
I have no clue even where to log in to admin or anything :)

(yes, it was silly of me not to stay on top of it and such - and it is not the end of the world - just slightly inconvenient)

5 Comments
2024/08/24
08:24 UTC

2

Flex, static files stopped working after upgrade

I've upgraded a PHP Yii2 on flex from 7.4 to 8.3 and it no longer serves the static files from subdirectories of my root folder. Anyone know why? The docs seem to suggest it should just work?

https://cloud.google.com/appengine/docs/flexible/serving-static-files?tab=php

runtime: php env: flex

runtime_config: operating_system: ubuntu22 runtime_version: 8.3 document_root: web

manual_scaling: instances: 1 resources: cpu: 1 memory_gb: 0.5 disk_size_gb: 10

2 Comments
2024/08/21
23:03 UTC

2

Flex environment network question

The pricing page says that network transfer is priced per compute engine internet egress rates. Does anyone know if it is priced per the premium network rates or the standard network rates and how can I change it?

1 Comment
2024/08/11
00:09 UTC

2

How would I clone project on a new computer

Hi there, probably a stupid question, but I've been working on a project for several years now, and the computer I'm using is probably not gonna last much longer.

What makes me worried is how I clone the app engine project to continue working from a new computer?

The project is also on GitHub, so can clone from there, but is the enough to tell app engine that it's the same project and same application?

4 Comments
2024/07/26
07:41 UTC

3

Standard Environment Works, But Flex Produces Errors...

I'm deploying my first application on App Engine and I'm having a deployment issue.

When I attempt to deploy it with the following app.yaml

runtime: nodejs20
env: flex

I get the following error

ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: Error(s) encountered validating runtime. Your runtime version for nodejs20 is past End of Support. Please upgrade to the latest runtime version available..

However, things work when I change my app.yaml like so:

runtime: nodejs20
env: standard

Any ideas why this is?

5 Comments
2024/07/19
14:58 UTC

1

New to AppEngine, looking for advice

Hey, so i am a complete beginner and am looking to take my locally hosted python website (using google service credentials) online. The site is purely for personal use amongst my friends and me, but would like the potential to go bigger eventually. As only 4 or 5 people would be using this website max for the time being, will i still have to pay?

My website is purely a search engine which pulls data from 6 different google sheets and displays images.

I would like to know details like -

How much usage would require me to pay?

Can i drag and drop my entire file containing my python flask app and all the rest (Index, function, style etc) into the AppEngine and it just works?

Are there lots more variables I don't know about?

Is AppEngine even the service i am looking for?

Any advice appreciated, the google cloud interface is super confusing to me. Thank you.

8 Comments
2024/03/25
09:46 UTC

2

What is the best cloud service for Telegram Bot on Python?

When considering Google Cloud Platform, which is better to choose Google Cloud VM, App Engine, Cloud Run or Dialogflow? And how exactly to do this, what is the deployment procedure, if possible, step by step? And also, which is better to choose, Polling or to use Webhook, and which libraries do you recommend to use?

For example, if we consider the following code:

import telebot
import google.generativeai as genai
bot = telebot.TeleBot("API KEY", parse_mode=None) # You can set parse_mode by default. HTML or MARKDOWN
genai.configure(api_key="API KEY")

# Set up the model
generation_config = {
  "temperature": 0.9,
  "top_p": 1,
  "top_k": 1,
  "max_output_tokens": 2048,
}

safety_settings = [
  {
    "category": "HARM_CATEGORY_HARASSMENT",
    "threshold": "BLOCK_MEDIUM_AND_ABOVE"
  },
  {
    "category": "HARM_CATEGORY_HATE_SPEECH",
    "threshold": "BLOCK_MEDIUM_AND_ABOVE"
  },
  {
    "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
    "threshold": "BLOCK_MEDIUM_AND_ABOVE"
  },
  {
    "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
    "threshold": "BLOCK_MEDIUM_AND_ABOVE"
  },
]

model = genai.GenerativeModel(model_name="gemini-1.0-pro",
                              generation_config=generation_config,
                              safety_settings=safety_settings)

convo = model.start_chat(history=[
  {
    "role": "user",
    "parts": ["Привіт!"]
  },
  {
    "role": "model",
    "parts": ["Привіт, чим можу допомогти?"]
  },
])

@bot.message_handler(func=lambda m: True)
def echo_all(message):
    convo.send_message(message.text)
    response = (convo.last.text)
    bot.send_message(message.chat.id, response)
      
if __name__ == '__main__':
    bot.infinity_polling()

Thank you!

0 Comments
2024/03/22
22:23 UTC

2

Local emulators / dev environment fails after java8 to java17 migration

Old dev env: java8 + eclipse + appEngine extension

New dev env: VSCode + dev containers + java17 + colima + docker + gcloud emulator datastore

I've successfully fumbled through the configuration and got my dev container to send requests to the emulator container... BUT... I get a java error about `memcache.GET()` not working or existing. Java is using objectify v5, I ensured `cache(false)` and I even upgraded to v6 and nothing seems to help.

I'm hoping someone has some ideas because I'm all out.

I have multiple services (formally modules) in AppEngine which all worked together in eclipse's appengine extension, so getting back to that would be ideal.

Questions:

  1. What are you all using for local emulators and setup?
  2. Should I abandon AppEngine and move to using open-source alternatives like `couchdb` instead of datastore, etc?
3 Comments
2024/03/09
05:49 UTC

3

Testing python webapps deployed on google app engine (GAE)

1 Comment
2024/02/14
08:58 UTC

3

Jan 2024 End of Support reminder (Python 2.x & 3.7, Java 8, Go 1.11-1.18, PHP 5 & 7, Ruby 2, Node 10-16)

REMINDER: "End of support" for the Python 2.x & 3.7, Java 8, Go 1.11-1.18, PHP 5 & 7, Ruby 2, Node 10-16 runtimes is coming at the end of this month. I posted about it recently on my Python/Google blog, but the tips/suggestions there mostly also apply to apps on any of the language/version runtimes listed in the title. To share with those affected, the specific post is offlinked from either LinkedIn or Twitter/X.

1 Comment
2024/01/05
01:29 UTC

4

Running php website on Google App Engine Standard doesn't route to subfolders and files nor can i go them directly

I'm trying to migrate our website from a linux vm (also on google) to google App Engine Standard enviroment.

When i deploy the app and test it the main page (index.php) works fine but when i try to go to other files, for example /somefolder/somefile.php it doesnt. It just shows the index.php but without the pictures etc.

I searched the internet and i found that this is probably due to not having a front end controller(?)

My app.yaml file is as followed:

service: nameoftheapp
runtime: php83

handlers:
# Serve images as static resources.
- url: /(.+\.(gif|png|jpg))$
  static_files: \1
  upload: .+\.(gif|png|jpg)$

- url: /(.+\.php)$
  script: auto

- url: /.*
  script: auto

my index.php is:

<?php



// android store
if (preg_match('#android#i', $_SERVER ['HTTP_USER_AGENT'])) {
    header('Location: market://details?id=nl.myapp');
    exit;
}

// ios
if (preg_match('#(iPad|iPhone|iPod)#i', $_SERVER ['HTTP_USER_AGENT'])) {
    header('Location: https://apps.apple.com/us/app/myapp/id973246494');
    exit;
}
echo "<html> <head> </head> <body>";
echo '<center><p><br><p><br>Sometext</center> <p>';
echo '<center> some more text.</center> <p>';
echo "<center> <img width='800' src='images/logo_GW_forweb.jpg'</center> <p>";
echo "<center> yet some more text</center> <p>";
echo "</body>";

?>

the index.php serves as a simple landing page for users to redirect them to the appstores for the app. As far as this goes, this works well. also the logo, which resides in a subfolder is shown.

But i myself want to go to https://mywebsite.nl/somefolder/somefile.php

This part doesnt work. Can this be resolved by just setting the right app.yaml (i do have like 10 subfolders with some having their own subfolders and a total of 100+ .php files)

Do i need something else? I was hoping there would be a settings for the app.yaml that routes all reguests to the right place.

I made a test app to see how to get it working. This one works, but i doubt this is the way to go.

The app.yaml file states:

runtime: php83
service: test

handlers:
- url: /.*
  script: index.php

And the index states:

<?php
$requestUri = $_SERVER['REQUEST_URI'];

// Hier kun je logica toevoegen om het verzoek te routeren naar de juiste actie/controller
if ($requestUri === '/test/test.php') {
    require 'test/test.php';
} elseif ($requestUri === '/root.php') {
    require 'root.php';
} else {
    // Standaard HTML-tekst als de URI niet overeenkomt met specifieke routes
    echo "<!DOCTYPE html>";
    echo "<html lang=\"en\">";
    echo "<head>";
    echo "    <meta charset=\"UTF-8\">";
    echo "    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">";
    echo "    <title>Test App</title>";
    echo "</head>";
    echo "<body>";
    echo "    <h1>Hallo dit is een test</h1>";
    echo "    <p>Welkom bij de PHP-testapplicatie op Google App Engine!</p>";
    echo "    <p><a href=\"test/test.php\">Ga naar test.php</a></p>";
    echo "    <p><a href=\"root.php\">Ga naar root.php</a></p>";
    echo "</body>";
    echo "</html>";
}
?>

This one works. i can access the root.php as well as the test.php which is located in the subfolder test. But i doubt this would be the way to go for my own website.

3 Comments
2024/01/02
11:12 UTC

2

mTLS Client Certificate Validation

I have a .net api hosted in GAE. I need to add mTLS to it. I've done this in Azure App Services where you simply tell it to require a cert and it gets passed to the app as a base 64 encoded header that the middleware then turns back into a certificate where you do your own validation. How would I accomplish this in GCP be it GAE or other service(s)?

0 Comments
2023/12/24
15:14 UTC

1

Need advice on developing back-end Stripe server

Hey all!

Sorry in advance- beginner developer here. I am practicing developing an “Airbnb”-like app in Xcode with SwiftUI and I’m having trouble incorporating a stripe payment backend. I need the ability to accept payments and payout hosts.

I’ve been referring to this documentation from Stripe: https://stripe.com/docs/connect/collect-then-transfer-guide#:~:text=responding%20to%20disputes.-,Payouts,on%20a%20daily%20rolling%20basis.

I’ve been trying to develop the server with Python through a virtual machine compute engine on Google Cloud. When I call the server using Alamofire through Xcode to create a connected user, the link returns as “nil” and the app freezes.

  1. Is running the Python server through a Google cloud compute engine a common way to set up the server?
  2. Before creating the connect account URL, the user needs a token?
  3. Is there any source code I could reference as an example?

I think the way I set up the server is the reason for these bugs and would appreciate any advice for where to set up the server to pass data to and from it.

I don’t want anyone to write the code for me, but any advice or documentation to help get over this hurdle for a beginner programmer would be greatly appreciated!

0 Comments
2023/11/06
03:11 UTC

2

Getting different country code in AppEngine request header X-AppEngine-Country for same IP address

I've a machine with fixed public IPv4 address. X-AppEngine-Country in the request header of api is sometimes showing US as country code and sometimes IN. No VPN is being used on this machine.

Has anyone faced similar issue in their appengine application?

Asked this question from AppEngine support but it seems they can't help because they do not read it from WHOIS database, as per their documentation

How to resolve the issue in such cases?

How to know if country returned by X-AppEngine-Country is correct or not?

0 Comments
2023/09/17
03:32 UTC

1

When should I deploy with Docker?

I am new to developing with App Engine and as the title indicates, I am wondering when it is preferable to deploy an application with docker.

Currently, I have a frontend built with Svelte/Sveltekit that is deployed to GAE. Similarly, I have an API built with FastAPI that is also deployed to GAE. To deploy these, as expected, I simply navigate to my app directories (e.g., "frontend" and "backend") and deploy them to App Engine with gcloud app deploy. However, I am seeing a lot of resources that are discussing applications that are deployed with a Dockerfile.

When is this useful? Is it something that I should be doing every time?

1 Comment
2023/09/08
00:26 UTC

2

Legacy app engine, Admin interactive dashboard in Python3 + Flask app

I need an alternative to /admin/interactive console, where we are able to run some code snippet in the service runtime.

Has anyone tried to build or know how to achieve the same in Python3 gae environment?

2 Comments
2023/08/02
10:48 UTC

1

Flask-Caching, Memecache, and App Engine. I could use some help/advice.

3 Comments
2023/07/14
17:09 UTC

2

Migrate Webapp2 framework to Python3 to use in App Engine Standard Environment v2

There is a need to migrate a huge codebase to App Engine Standard v2 + Python 3 from App Engine Standard v1 + Python 2.

I'm trying to write a wrapper around Flask web framework that should ideally require only change in import statement of webapp2 inside a module. But it doesn't seem that this approach will handle all the cases.

Here is link to webapp2 https://cloud.google.com/appengine/docs/legacy/standard/python/tools/webapp2

Since codebase is quite huge with complicated dependency on webapp2, I'm trying to find a way to make webapp2 itself work in App Engine Standard v2 environment with Python 3.

Refactoring codebase to use Flask directly will not be feasible for us at the moment, as this would be really slow process to migrate our code to App Engine standard v2.

Has anyone tried this approach, instead of moving to Flask?

Could anyone please suggest how this can be achieved?

5 Comments
2023/07/08
10:31 UTC

1

Read custom non-standard headers in Gunicorn + Flask in AppEngine Standard Environment version 2

A client of my app needs to send some headers with underscore in them, e.g.

x_email: test@gmail.com

x_some_custom_header: somevalue

My service is running using the stack Gunicorn + Flask in AppEngine Standard Environment.

As seen in the logs, app engine starting an Nginx reverse proxy in front of Gunicorn.

And this is a known default configuration of Nginx which do not allow headers with underscore in their name.

Any one has encountered such issue, how you solved it in App Engine Standard V2?

4 Comments
2023/06/20
11:13 UTC

5

Surprise bill for cloud storage

I recently started developing an app on appengine in the standard environment. I've developed a few apps in the past on the older version of the standard environment, so I wanted to see what the newest one was like and what it had to offer. I was a bit surprised to get a bill at the end of last month after only two weeks of developing my app. The bill is only for a couple of cents, but it's definitely surprising considering that I expected everything to be within the free tier and I never had this experience with the previous apps I built.

From what I can see of the cost breakdown, the cost is for "Cloud Storage" and specifically "Networking Traffic Egress GCP Replication within Northern America". My app isn't doing anything with Cloud Storage, so it's surprising to get a bill for this service. Looking at the buckets within Cloud Storage reveals a number of what I can only assume are layers to build the image for my app.

  • Is there something I can do to prevent this charge?
  • Is this called out in the documentation somewhere?
  • If there is a way to turn this feature off, why isn't that the default?
2 Comments
2023/04/04
01:48 UTC

1

Pros and Cons of Mapping Custom domain

Just finished migrating a Spring boot app I am hosting on my home server to GAE. The main driver for this effort was the unreliability of Xfinity (my ISP). I was saving some money hosting my own app, but soon realized the value of hosting it on the cloud and not having to manage a server, keeping up with security, patches, hardware failures, etc.

I'm currently using a custom domain that is managed by Cloudflared, expiration is approaching around June.

Since this is a hobby app, I was looking at the possibility of just using the app engine generated URL and letting my custom URL expire.

I was wondering if I can get feedback from anyone who has chosen to keep the app engine generated URL. I was searching prior to this post but could not info much info.

For starters, I'm wondering if there are any issues from the Search Engine perspective. In this aspect I do see the value of keeping the custom domain since it has been search/indexed by search bots. But, I don't really have much traffic. This is what drove me to GAE because I can have the 0 instances running.

2 Comments
2023/01/12
22:44 UTC

2

Exceeded soft memory limit of 256 MiB...

Want to keep this brief and hopefully not need to go to stackoverflow:

Here are my logs histogram, as you can see I'm getting an error of this type about once an hour, and some intermittent warnings, but it's not the lions share of the service. I recently learned that this has been happening since early December, and increasingly so. I figured it was just an issue of inefficient code (Python/Flask), refactored my index page, but it's still happening:

Exceeded soft memory limit of 256 MiB with 280 MiB after servicing 956 requests total. Consider setting a larger instance class in app.yaml.

293 MiB after servicing 1317 requests

260 MiB after servicing 35 requests

The strange thing is that it's happening on pages like

/apple-touch-icon.png

that should 404.

Here are some other things that may be causing the problem. First my app.yaml page has settings that I added before my site was as popular that are extremely lean to say the least:

# instance_class: F1 (default)

automatic_scaling:
    max_instances: 3
    min_pending_latency: 5s
    max_pending_latency: 8s
    #max_concurent_requests: 20
    target_cpu_utilization: 0.75
    target_throughput_utilization: 0.9

The small instances, min and max latency, and cpu utilization are all obviously set for slower service, but I'm not made of money, and the site isn't generating revenue.

Secondly, looking at the logs recently, I'm getting absolutely slammed by webcrawlers. I've added them to robots.txt:

User-Agent: MJ12bot
Crawl-Delay: 20
User-Agent: AhrefsBot
Crawl-Delay: 20
User-Agent: SemrushBot
Crawl-Delay: 20

It looks like all but Semrush have died down a bit.

Anyway, thoughts? Do I just need to upgrade to F2, or is there something in the settings that I've definitely got wrong.

question on stackoverflow

1 Comment
2023/01/11
20:01 UTC

1

React app logs on Google App Engine says "Starting the development server..." and takes 30+ seconds to start

I have deployed a React app to Google App Engine (using an F2 instance, nodejs16 runtime) and the start up is incredibly slow, at least 30 seconds. I've also noticed in the logs that it says "Starting the development server...", which seems bad because it's obviously in production.

Are the two things related? And, either way, should I be doing something so that a production server is being started? Before deploying, I did run npm run build and so the build folder has been included in the deployment

0 Comments
2023/01/06
22:09 UTC

2

Cloud tasks URL and filtering

I'd like to be able to specify a filter in the URL to a cloud tasks queue page to make it easier for developers to query particular tasks. It looks like the filter is done using Javascript and a separate HTTP request. Is there a way to do something like this:

https://console.cloud.google.com/cloudtasks/queue/<location>/<queue name>/tasks?project=<project-id>&filter=someFilter

I.e. to be able to specify the filter in the URL.

0 Comments
2023/01/05
23:26 UTC

2

Restrict domain origin

Hi,

I've got a flask app running in standard mode. I built it for a client and I've agreed to host it. However, I want to lock it down so it's accessible only by a user originating from their domain. I tried flask-cors, but no joy. IS there a way to do this with the app.yaml.

Example: only allow user access the to the web page www.mydomain.com/registration if they originate from www.customerdomain.com

7 Comments
2022/11/30
16:03 UTC

1

Database viewer for local Spanner emulator

I use the local Spanner emulator for local testing of a Java application. I'm struggling to find a local application that will let me view the configuration and contents of the database. E.g. to run adhoc queries etc to aid with development. I'd like this to be in a docker container if possible (like pgAdmin).

Does anyone have any tips on how to do this?

3 Comments
2022/11/27
22:49 UTC

Back To Top