Appengine - Deployment of hidden folder

Question

To verify a SSL certificate, I need to upload a hidden folder ("/.well-known" containing some files to my application.

I am deploying java application with eclipse, but these files do not receive at the application on appengine. I guess they are filtered out.

I tried to add the hidden folder as static file to the appengine-web.xml, but it did not help.

<!-- Configure serving/caching of GWT files -->
<static-files>
    <include path="**" />
    <include path=".**" />
    <include path="**.*" expiration="0s" />
    <include path="**.well-known" expiration="0s" />
    <include path="**.nocache.*" expiration="0s" />
    <include path="**.cache.*" expiration="365d" />
    <include path="**.css" expiration="30d"/>
    <exclude path="**.gwt.rpc" />
    <exclude path="**.html" />
</static-files>

Any ideas how I could upload these folder and the files?

Darn

Answer 1

For anyone else coming here like me after trying to serve the challenge for letsencrypt in a static manner in Google App Engine and failing, the following did it for me: (one might be able to actually do it statically, but I didn't try it as I didn't want to spend more time trying out stuff and Ian apparently tried that and could not make it work [maybe the copy command done internally on Google App Engine ignores the directories that start with a dot] )

Taken from http://igorartamonov.com/2015/12/lets-encrypt-ssl-google-appengine/ credits go to Igor Artamonov.

Just build a servlet like:

public class LetsencryptServlet extends HttpServlet {

    public static final Map<String, String> challenges = new HashMap<String, String>();

    static {
        challenges.put("RzrvZ9gd7EH3i_TsJM-B0vdEMslD4oo_lwsagGskp6c",
                "RzrvZ9gd7EH3i_TsJM-B0vdEMslD4oo_lwsagGskp6c.ONrZa3UelibSWEX270nTUiRZKPFXw096nENWbMGw0-E");
    }

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp)
            throws ServletException, IOException {
        if (!req.getRequestURI().startsWith("/.well-known/acme-challenge/")) {
            resp.sendError(404);
            return;
        }
        String id = req.getRequestURI().substring("/.well-known/acme-challenge/".length());
        if (!challenges.containsKey(id)) {
            resp.sendError(404);
            return;
        }
        resp.setContentType("text/plain");
        resp.getOutputStream().print(challenges.get(id));
    }
}

And add to web.xml somethine like:

<servlet>
    <servlet-name>letsencrypt</servlet-name>
    <servlet-class>...LetsencryptServlet</servlet-class>
</servlet>
<servlet-mapping>
    <servlet-name>letsencrypt</servlet-name>
    <url-pattern>/.well-known/acme-challenge/*</url-pattern>
</servlet-mapping>

Of course, be sure that the servlet class has the full classpath for your created Servlet.

That blog post also deals with the other steps necessary to generate and install the certificate.

Ian: Are you sure that you were deploying the servlet well? check the logs, make sure that you are testing the right version.. maybe you had a compilation issue..

Cheers

Quill

Did it the same way (see my anwer below). Thanks for the detailed instructions! – Darn

Answer 2

I ran into this problem trying to serve an assetlinks.json file. It would indeed appear that folders starting with a . are not accessible within the static context in App Engine. A more generic version of João Antunes workaround is as follows.

First, create the folder without the . at the start and place any required files inside it.

We then need to create a servlet that will respond with the correct data when a request to the hidden folder is received.

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.InputStream;

/**
 * Created by Will Calderwood on 17/05/2017.
 * <p>
 * It would appear to not be possible to upload hidden folders to app engine. So when files need
 * to be served from a hidden folder the URL can be bounced through this servlet
 */
public class StaticFileServer extends HttpServlet {
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        // We'll remove the dots from the path
        String uri = req.getRequestURI().replace("/.", "/");

        // Do anything else that needs doing here
        if (uri.toLowerCase().contains(".json")) {
            resp.setContentType("application/json");
        }

        // Read and return the resource from the non-hidden folder
        try (InputStream in = getServletContext().getResourceAsStream(uri)) {
            if (in == null){
                resp.sendError(404);
                return;
            }
            byte[] buffer = new byte[8192];
            int count;
            while ((count = in.read(buffer)) > 0) {
                resp.getOutputStream().write(buffer, 0, count);
            }
        }
    }
}

Then add the following to your web.xml file to point the hidden folder at our servlet

<servlet>
    <servlet-name>StaticFileServer</servlet-name>
    <servlet-class>main.com.you.StaticFileServer</servlet-class>
</servlet>
<servlet-mapping>
    <servlet-name>StaticFileServer</servlet-name>
    <url-pattern>/.well-known/*</url-pattern>
</servlet-mapping>

Tumultuous

Answer 3

I did find a better way, the issue is that the code is stored in a hidden directory, so we just need to map the correct url to a visible directory.

handlers: - url: /.well-known static_dir: _well-known

Dionysiac

I'm not sure there's an equivalent in the Java version. – Tumultuous

I got here while searching for solution for Python, eventually found it. The option is named skip_files and you can override it. The last pattern is the one for hidden files, just delete it and paste the rest. – Hogen

Answer 4

Static files are not uploaded to the application servers and thus are not accessible by app code. From appengine-web.xml Reference, in the <static-files> section:

< static-files >

Optional. The < static-files > element specifies patterns that match file paths to include and exclude from the list of static files, overriding or amending the default behavior. Static file are served from dedicated servers and caches that are separate from the application servers and are useful for serving static content such as images, CSS stylesheets or JavaScript files.

To have files accessible by app code they need to be specified in the <resource-files> section (same doc as above):

< resource-files >

Optional. The files that listed in the < resource-files > element are accessible by the application code using the filesystem. These files are stored on the application servers with the app as opposed to how static files are stored and served.

Condescend

I tried to mark these files as static files, so they would be accessed from the outside (see my code snipped). Maybe I did the syntax wrong, but they are not served by appengine after deployment. – Darn

Not being served could also mean they're not accessed properly or not uploaded in the right place. What do you see in the logs for when they're not served? I'd 1st try to download the deployed app code, to verify if the dir was actually uploaded (in the right place) or not. If it was uploaded focus goes into accessing the files. If not it goes into getting the file uploaded. – Condescend

If I run that app locally, I can access the file on the local host, after deploying, I replaced the localhost with my domain-URL, and it gives me a 404. So it seems it is either not uploaded or it is not served. – Darn

Any news on this @jan? – Deenadeenya

@João Antunes I solved it the same way you did it: With a servlet. Thanks for the detailed response! – Darn

Answer 5

I tried everything, but could not manage to upload that hidden folder. I then defined a servlet mapping for that path an the servlet writes out the verification string expected bei the SSL issuer:

<servlet-mapping>
    <servlet-name>SSLVerificationServlet</servlet-name>
    <url-pattern>/.well-known/acme-challenge/roQH_vUmRQsuKH8lZGedft9O6TK-UoZWzv_kY8MukH4F8</url-pattern>
</servlet-mapping> 

Darn

Have you tried this: igorartamonov.com/2015/12/lets-encrypt-ssl-google-appengine ? – Deenadeenya

Answer 6

An ugly, but straightforward workaround is to use JSP support to map the page. One example:

Create your json file in package/src/main/webapp/_well-known/assetlinks.json.jsp

<%@ page contentType="application/json;charset=UTF-8" language="java"%>
[JSON CONTENT]

Map the "servlet" to the desired path in web.xml:

<servlet>
    <servlet-name>Asset Links</servlet-name>
    <jsp-file>/_well-known/assetlinks.json.jsp</jsp-file>
</servlet>
<servlet-mapping>
    <servlet-name>Asset Links</servlet-name>
    <url-pattern>/.well-known/assetlinks.json</url-pattern>
</servlet-mapping>

Done!

Reefer