Life has exceptions, and Web Applications are no Exception

I started writing this post as a bug-tracker ticket, but I soon realised that the issue is so generic that I decided to make it a public post instead.

The problem

So, the prolem is this: we have a web application, obviously (nowadays), that consists of client code that gets data and other services from web API invocations. I’m usually the one developing the server side (God, protect me from end users and Javascript, I can take care of the server side…).

When calling the API, the client should check if the returned HTTP status is in the 200 – 299 range, it should not try to process the output data as if the API succeeded.

In particular, failing this check can cause data processing errors that the end user doesn’t see, leaving them with a blank screen and the feeling the UI is broken without knowing why, which also causes not useful error reports (‘the screen is blank’, rather than ‘the error <X> happened’).

For instance, this code trying to get a product ID will break if the response contains some JSON about an error (or worse, some HTML about it):

// Courtesy of ChatGPT
fetch("/api/products/123")
  .then(response => response.json())  // even if 404 or 500!
  .then(data => {
    // Suppose we expect { id, name, ... }
    const productId = data.id; 
    renderProduct(productId, data.name);
  })
  .catch(error => {
    // Good, but misses bad HTTP responses
    console.error("Network error:", error);
  });

The general fix

To address this, you should:

1. Make client code aware that API calls can fail and halt the flow when they do.
2. have a general top layer exception interceptor (ie, which is triggered when no other flow does a catch exception {...} to try its own recovering), which should interrupt the regular flow and should tell the user something bad has happened.

The above example could be fixed with something like:

fetch("/api/products/123")
  .then ( response => {
    if (!response.ok)
      throw new Error(`HTTP ${response.status}: ${response.statusText}`);

    return response.json();
  })
  .then(data => {
    const productId = ...
  })
  .catch( error => {
    console.error("Request failed:", error);
    showUserError("Sorry, we couldn't load the product details, due to the data service error:", error );
  });

With the API contribution

Here, the API has some good behaviour to express too: set up an error handler on the web requests, which format a (JSON) nice report to return in place of the expected data (and of course, return a proper HTTP status code).

Not only do most web frameworks provide helpers to ease this, but several conventional approaches for reporting web errors have also been proposed. In my APIs, I’ve adopted a variant of RFC 9457 and if you work with Spring, here it is the reusable handler for that.

So, once you have a behaving API, a client API error handler should exploit the error report that the API returns when HTTP Status != 200 (see details below) to report the generic short title coming from such API report and logging (console.error()) the details, which are included in the same report (for the benefit of developers or advanced users, server developers should make sure sensitive information don’t escape this way).

CORS complications

Another problem that unchecked HTTP status code causes to client developers is the false impression that they have CORS problems. As it is well-known, when some Javascript from a given site tries to get data from another site (as it normally happens in an API call, especially if you have some degree of microservice architecture), the browsers block the cross-site request by default, due to security reasons. API developers usually solve this problem by attaching an HTTP header to the API invocation’s output, which tells the browser to disable this restriction (or, in a few cases, it tells them to disable it for a list of known web domains).

However, this can only happen at the API implementation level, not before that is reached. Unfortunately, you typically have an upfront generic web server, such as NGINX or Apache, which cannot relax the restriction for all the requests it gets (technically possible, but too insecure, it could be done more selectively, by filtering the URLs about APIs, but too time-consuming). So, if a request yields errors like "404/URL Not found" or "500/ the internal API server is down", it’s the upfront web server that replies to the client, without anyone setting the CORS-disable header.

On top of this situation, if you try to process a request that has failed this way pretending it didn’t fail, you will see the CORS blocked error, which is not the actual error that happened. In order to know the latter, you need to check the HTTP status code returned from the API invocation and you need to handle an error situation as explained above.

Typically, a web server will return HTML in this case (the default error page), but the client can still: 1) know an error happened 2) check if there is something like an RFC 9457 error output 2.1) if yes, use the output to report error details 2.2) if no details are available, use the HTTP status code to tell the end user a more generic error message (most programming languages have libraries that map HTTP codes to human-readable descriptions).

Happy coding, even when it’s not happy

In conclusion, since life doesn’t always go as good as we wish and software isn’t infallible (remember that Murphy guy?), don’t code as if exceptions never occur or every component always returns valid output code.

For if you do, sooner or later you’ll inevitably face the consequences, and worse, you might not see the actual reason why you’re facing them.

Bonus: my format for reporting exceptions from Java web API.

As said above, I’ve a Java component that I attach to Spring applications, so that it catches all unhandled exceptions, does some automation to establish the right HTTP Status output and yields error reports in RFC 9457 format. For the latter, I’ve adopted a customised variant:

{
   // The FQN of the Java exception's class (top level)
   "type": "org.springframework.http.converter.HttpMessageNotReadableException",

   // Generic message
   "title": "Failed to read request",

   // HTTP status (returned the usual HTTP way too)
   "status": 400,

   // Detailed message (ie, the Java's exception message)
   "detail": "JSON parse error: Unexpected end-of-input: expected close marker for Object (start marker at ...",

   // the endpoint, as per RFC-9457
   "instance": "/v1/graph-info/elements",

   // The Java stack trace (omitted by default, see the configuration options)
   "trace": "org.springframework.http.converter.HttpMessageNotReadableException: ...\n
     at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType..."

}

 Bonus: example of putting all together

This shows how to report RFC-9547 output like the above with more generic bad responses:

// async/await version
fetch("/api/products/123")
  .then(async response => {
    if ( response.ok ) return response.json();
    // Try to parse as RFC 9457 problem detail
    let problem;
    try {
      problem = await response.json();
    } catch {
      // Not RFC 9457? Report a generic error
      problem = {
        title: response.statusText,
        detail: `HTTP ${response.status}`,
      };
      throw problem;
  })
  .then(data => {
    const productId = data.id;
    renderProduct(productId, data.name);
  })
  .catch(error => {
    const userMessage = error.title || "Unexpected error";
    const devDetails  = error.detail || error.toString();

    showUserError(userMessage);
    console.error(devDetails);    // diagnostics for developers
  });