Learning by Doing: HackerNewsSearch

One of my recent widgets, HackerNewsSearch, was an AngularJS app I wrote to practice handling asynchronous http requests to an API that may be unreliable, or maybe the Internet connection is slow or unreliable. The source is available here: https://github.com/xerocross/hacker-news-search.

That said, the front-facing activity of this app is not very useful. It queries Hacker News for top stories, lists then, and then offers a form up top you can use to search within the titles. It’s a very dumb search form. It filters out everything that doesn’t have the exact search string as a substring. But that activity was not really the point.

To mimic a slow internet connection and/or an unreliable server, I wrote a very simple proxy for the Hacker News API. I wrote my own tiny server using node and express. When you send it a get request, my server queries the Hacker News API. Then it takes its sweet time about responding. The delay is randomized, and like a third of the time it just sends a server error response. (I don’t think my API will get much use outside my own little project.)

Next I needed to write an AngularJS service for querying this unreliable API. At first, I my idea was just to wrap Angular’s $http.get function inside my own get function that will try the get request a few times with a delay in between (maybe 1500 ms), and resolve a promise reflecting either that it finally succeeded or that all attempts failed. I used the $q service to wrap these $http promises into my own custom-made promise, and that was the return value of my get function.

Having something like that allowed me to show a “loading” message while the service was trying (over and over) to hit the API, and then show either the data on success or a failure message if all attempts failed.

That would be fine, but after playing with that some I realized that just showing a “loading” message for possibly 10 seconds is not a good user experience—even if you throw in a spinning ball that gives you no information about progress. I wanted to somehow give the user progress updates, like “first attempt failed; retrying”. Promises do not offer that kind of data exchange. I’m sure I could have built something that did that using various old tools. But then I realized I recently learned about a tool that does exactly this sort of thing: RxJS Observables.

The Observable paradigm is not built into AngularJS the way it’s built into Angular 2+. But you can still use it. I wrote a get function like what you see here:

this.get = function(url) {
        let iteration = 0;
        let observable = new Observable((observer)=> {
            let attempt = function() {
                if (iteration >= numTries) {
                    observer.next({
                        status: "FAIL"
                    });
                } else {
                    observer.next({
                        status: "ATTEMPTING",
                        data : {
                            attemptNum : iteration
                        }
                    });
                    iteration++;
                    $http.get(url)
                    .then((response)=>{
                        if (response.status == 200) {
                            observer.next({
                                status: "SUCCESS",
                                data : response.data
                            })
                        } else {
                            attempt();
                        }
                    })
                    .catch((e) =>{
                        attempt();
                    })
                }
            }
            attempt();
        })
        return observable;
    }

I’ll go through it. You could use something similar in your own code, but some of this might be specific to my use case. As you can see, my get function is still basically a wrapper around the $http.get method, but all that happens inside the subscribe function of a new Observable. Whenever the attempt function is executed, as long as we have not run out of attempts, we add an “ATTEMPTING” message to the Observable’s data stream, which includes the current attempt number. Then we launch an $http.get request. The $http.get method returns a promise, so we have to handle that with .then and .catch. Together, these will handle all possible outcomes from the promise. The only successful response will be a status = 200 (because this is a get request). If anything else happens, we launch a new attempt. If we get a status = 200, then we push a “SUCCESS” message onto the data stream long with the response data from the http request. If all attempts fail, then we push a “FAIL” message to the data stream.

With this mechanism in place, on the client side we can update a user-facing update message for each of these messages coming from the Observer’s stream of data. If we are on attempt number 2, we can inform the user that a problem occurred but we are still trying. If all attempts failed, we can at least tell the user “we tried several times to fetch this data, but we could not connect to the server”—or something less wordy.

I find that this works well. In my interface, I implemented more informative loading and error messages, and it makes a better user experience. Moreover, with multiple tries to hit the API, this get request is more durable against little network outages. I could make some improvements probably. Maybe I should implement an increasing time delay between the attempts to give the network connection more time to repair itself in the event of a brief network failure. It would be simple enough to make that change.

To handle things like post requests, we could write a wrapper function very similar to the one I wrote for $http.get. For a post, we would expect a different status code to reflect the post was successful, but otherwise the internal logic would be the same: maybe. There might be implementation details I haven’t though of because I haven’t actually done it. Eventually I should add a post method to this project.

In the end, if you look at this widget, hosted on WidgetWonk, you will see it slowly, painfully pull the list of Hacker News top stories from my unreliable proxy server. Most of the time, each get requests eventually succeeds, but if one of them fails all attempts then you get a nice button you can click to try again.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s