Friday, July 17, 2009
HttpHelper
It's been a week of tweaking, fixing little bugs, spiralling in towards hoped-for perfection. I'm pleased to report that I now have an alpha-test version of my iPhone app up and running on my iPod Touch, and it looks to work a charm. If all goes very well indeed I might submit it to the App Store - where it will ultimately be available for the low low price of $0.00 - by the end of the month.
In the meantime, here's a utility class that people might find handy: my HttpHelper singleton. It's not really a class variable, since it seems Objective-C doesn't support them (Oh, Smalltalk, how I miss your class instance variables) but "static" has the same effect, if you code carefully. Like so:
//
// HttpHelper.m
// iTravelWrite
//
// Singleton class used to send HTTP requests and forward responses to selectors passed in by the caller.
// Created by Jon Evans on 04/07/09.
//
#import "HttpHelper.h"
#import "Util.h"
@implementation HttpHelper
static HttpHelper *singleton=nil;
#pragma mark -
#pragma mark Singleton methods
+(HttpHelper *) getInstance {
if (singleton==nil)
singleton = [[HttpHelper alloc] init];
return singleton;
}
+(id)allocWithZone:(NSZone *)zone {
if (singleton == nil) {
singleton = [super allocWithZone:zone];
return singleton;
}
return nil;
}
-(id)copyWithZone:(NSZone *)zone {
return self;
}
-(id)retain {
return self;
}
-(unsigned)retainCount {
return UINT_MAX;
}
-(void)release {
//pass
}
-(id)autorelease {
return self;
}
- (void)dealloc {
[super dealloc];
}
All the above is housekeeping stuff to ensure that we only ever have one instance of an HttpHelper. Not that a duplicate would be so disastrous in this case, but hey, if you're writing a singleton, write a singleton, right?
Here's the part where it actually does stuff. In particular, it does all the HTTP GETs and HTTP POSTs that your iPhone app will ever need:
#pragma mark -
#pragma mark Business logic
+(NSURLRequest*) buildRequestWithPostKeys:(NSArray *) postKeys postValues:(NSArray *) postValues urlString:(NSString *)urlString {
NSMutableString *params=[[NSMutableString alloc] initWithCapacity:1024];
for (int i=0; i<[postValues count]; i++) {
[params appendString:[postKeys objectAtIndex:i]];
[params appendString:@"="];
[params appendString:[postValues objectAtIndex:i]];
[params appendString:@"&"];
}
NSData * paramData = [params dataUsingEncoding:NSUTF8StringEncoding];
NSURL * url = [NSURL URLWithString:urlString];
NSURLRequestCachePolicy policy = NSURLRequestReloadIgnoringCacheData; // never cache a post response, at least in my app
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url cachePolicy:policy timeoutInterval:10.0];
NSString *msgLength = [NSString stringWithFormat:@"%d", [paramData length]];
[request addValue: msgLength forHTTPHeaderField:@"Content-Length"];
[request setHTTPMethod:@"POST"];
[request setHTTPBody: paramData];
return request;
}
+(BOOL) doPost:(NSURLRequest *)request forCaller:(id)caller onSuccess:(SEL)onSuccess onFailure:(SEL)onFailure {
NSArray *keys = [NSArray arrayWithObjects:@"request", @"caller", @"onSuccess", @"onFailure", nil];
NSArray *values = [NSArray arrayWithObjects:request, caller,
[NSValue valueWithBytes:&onSuccess objCType:@encode(SEL)],
[NSValue valueWithBytes:&onFailure objCType:@encode(SEL)],
nil];
NSDictionary *args = [NSDictionary dictionaryWithObjects:values forKeys: keys];
NSThread* uploadThread = [[NSThread alloc] initWithTarget:[self getInstance] selector:@selector(doHttp:) object:args];
[uploadThread start];
[uploadThread release];
return TRUE;
}
+(NSURLRequestCachePolicy)getCachePolicyFor:(NSString *)urlString {
/*
Hived out to a separate method because we might fine-tune this later.
In theory, this will cause the app to use the cache policies set by
wetravelwrite.appspot.com, which as of this writing means 1 hour
for listing information, and 10 hours for searches.
In the ListingEditController and ListingsViewController we manually wipe
the cache for listings edited by the app.
*/
// return NSURLRequestUseProtocolCachePolicy;
return NSURLRequestReloadIgnoringCacheData;
}
+(NSURLRequest *)getURLRequestFor:(NSString *)urlString {
NSURL *url = [NSURL URLWithString:urlString];
NSURLRequestCachePolicy policy = [self getCachePolicyFor:urlString];
NSURLRequest *request = [NSURLRequest requestWithURL:url cachePolicy:policy timeoutInterval:7.5];
return request;
}
+(BOOL) doGet:(NSString *)urlString forCaller:(id)caller onSuccess:(SEL)onSuccess onFailure:(SEL)onFailure {
NSURLRequest *request = [self getURLRequestFor:urlString];
NSArray *keys = [NSArray arrayWithObjects:@"request", @"caller", @"onSuccess", @"onFailure", nil];
NSArray *values = [NSArray arrayWithObjects:request, caller,
[NSValue valueWithBytes:&onSuccess objCType:@encode(SEL)],
[NSValue valueWithBytes:&onFailure objCType:@encode(SEL)],
nil];
NSDictionary *args = [NSDictionary dictionaryWithObjects:values forKeys: keys];
NSThread* uploadThread = [[NSThread alloc] initWithTarget:[self getInstance] selector:@selector(doHttp:) object:args];
[uploadThread start];
[uploadThread release];
return TRUE;
}
-(BOOL) doHttp:(NSDictionary *)args
{
@synchronized (self) {
//autorelease pool
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSURLRequest *request = [args objectForKey:@"request"];
NSObject *caller = [args objectForKey:@"caller"];
SEL onSuccess;
[[args objectForKey:@"onSuccess"] getValue:&onSuccess];
SEL onFailure;
[[args objectForKey:@"onFailure"] getValue:&onFailure];
NSError *error;
NSData *returnData = [NSURLConnection sendSynchronousRequest: request returningResponse: nil error: &error ];
NSString * responseString = [[NSString alloc] initWithData: returnData encoding: NSASCIIStringEncoding];
if (!error) //connection succeeded; but did it work?
error = [Util getErrorFrom:responseString forRequest:request];
if (error) {
NSLog(@"iTravelWrite Error ",error);
[caller performSelectorOnMainThread: onFailure withObject:error waitUntilDone:NO];
}
else
[caller performSelectorOnMainThread: onSuccess withObject:responseString waitUntilDone:NO];
[pool release];
}
return TRUE;
}
@end
Pretty slick, huh?
The three chief interface methods are, for a GET -
+(BOOL) doGet:(NSString *)urlString forCaller:(id)caller onSuccess:(SEL)onSuccess onFailure:(SEL)onFailure
and for a POST -
+(NSURLRequest*) buildRequestWithPostKeys:(NSArray *) postKeys postValues:(NSArray *) postValues urlString:(NSString *)urlString
+(BOOL) doPost:(NSURLRequest *)request forCaller:(id)caller onSuccess:(SEL)onSuccess onFailure:(SEL)onFailure {
I'll give you an example of the former first, as it's easier. The call itself is perfectly straightforward:
NSMutableString *urlString = [NSMutableString stringWithCapacity:128];
[urlString appendString:[Util getSearchPageURL]];
[urlString appendString:@"?locale="];
[urlString appendString:[UserSettings getLanguage]];
[urlString appendString:@"&searchTerms="];
[urlString appendString:[searchBar.text stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]];
[HttpHelper doGet:urlString forCaller:self onSuccess:@selector(parseSearchResults:) onFailure:@selector(searchFailed:)];
Not, however, the two "@selector" arguments. These must be methods on the caller, and they better expect an NSString* and an NSError*, respectively, as arguments. (Or they can take an NSObject* and cast from there, I suppose, but why bother, right?)
Inside HttpHelper we have to wrap the selectors in NSValues to pass them from method to method, which is a bit annoying, but hey, we only have to do it once.
To call a POST, by comparison:
NSArray *postKeys = [NSArray arrayWithObjects:@"title", @"location", @"comments", @"pageUri", @"sectionName", @"sectionNumber", @"listingName", nil];
NSArray *postValues = [NSArray arrayWithObjects:note.title, [note locationString], note.body, note.pageUri, note.sectionName, note.sectionNumber, note.listingName, nil];
NSURLRequest *request = [HttpHelper buildRequestWithPostKeys:postKeys postValues:postValues urlString:[Util getUploadNoteURL]];
[HttpHelper doPost:request forCaller:self onSuccess:@selector(onUploadSuccess) onFailure:@selector(onUploadError:)];
Note that "onUploadSuccess" here doesn't take an argument - I don't care about the web site's response, the fact of success is all that matters. OK, dubious wisdom that, but the example is relevant to show that the selector methods don't actually have to accept arguments.
We launch a new thread every time we call HttpHelper, so we have to be careful lest we run into concurrency problems. So we synchronize the actual HTTP calls in the "doHttp:" method, which is the one place in the app where we actually go out and connect to the big bad scary Internet.
Note also that if we wanted to change from using "sendSynchronousRequest:" to the delegated version of NSURLConnection, the only class that would change is HttpHelper. Ah, encapsulation.
Anyway. Share and enjoy, as the man said. Hope all that's useful to someone out there...
Labels: GET, HTTP, HttpHelper, iPhone, NSRequest, NSThread, NSURL, NSURLConnection, Objective-C, POST, SDK, selector, singleton
Wednesday, June 17, 2009
I am the alpha dog!
Mind you, the Android app is a fairly trivial piece of programming next to the AppEngine middleware that does the actual parsing and updating of WikiTravel. Which is the way it should be. Resources are scarce on a smartphone, and both processing and battery power are relatively meagre; I expect the design pattern of choice is going to be "get as much of the work done as you can on your server farm, then communicate simple low-bandwidth stuff to the phone."
Ultimately, the phone isn't much more than a user interface. In fact, now that I think about it, my whole app follows the classic Model-View-Controller design: WikiTravel is the model, the smartphone is the view, and my AppEngine service is the controller. Huh. Plus ça change, plus c'est la même chose.
Anyway, a list of tips, tricks, and annoyances since last we met:
- Let's start off with the annoyances. Character encoding. This has never been my strong suit, and I all too often wind up trial-and-erroring a solution. What doesn't help here is that I've got URL encoding (eg "?"->"%3F" in an HTTP request), HTML encoding (eg & -> & in a web page) and actual character encoding (ASCII? Unicode? UTF-8?) and while I grok the first two, I always have a mental block with the latter. I seem to have brute-forced something like a solution. We'll see. I expect it will rear its ugly head again.
Python, weirdly, for all its text-processing power, doesn't come with much in the way of built-in HTML escaping and unescaping. "cgi.escape" will do much, but, annoyingly, not everything in the way of escaping. As for unescaping, fuhgedaboudit. Fortunately, I found this neat little solution somewhere on the Internets:
#HTML unescape
#taken from the Internet
class Unescaper:
entity_re = re.compile(r'&(#?[A-Za-z0-9]+?);')
def replace_entities(self,match):
try:
ent = match.group(1)
if ent[0] == "#":
if ent[1] == 'x' or ent[1] == 'X':
return unichr(int(ent[2:], 16))
else:
return unichr(int(ent[1:], 10))
return unichr(name2codepoint[ent])
except:
return match.group()
def html_unescape(self,data):
return self.entity_re.sub(self.replace_entities, data) - While I'm complaining, as a Java coder, I'm used to all objects having a "toString" representation, and thus, when logging or debugging, being able to put
"this is"+anObject+" and this is "+anOther
without worrying about syntax. But Python, otherwise a far superior text-processing lanaguage, won't let you do this: you have to put "str()" or "repr()" around objects you want to include in a string. For no apparent reason. Sigh. - The good news is, the Java SDK now includes a perfectly acceptable HTML parser. I needed to parse attributes in a tag, didn't want to include a whole new JAR in my project for just that purpose, and really didn't want to write that code myself (it'd be like reinventing the wheel in Detroit.) org.xml.sax to the rescue:
class ListingHandler extends DefaultHandler {
public void startElement(String namespaceUri, String localName, String qualifiedName,
Attributes attributes) throws SAXException {
for (int i=0; i<attributes.getLength(); i++) {
String field = attributes.getLocalName(i);
String value = attributes.getValue(i);
and you don't even have to fuss with XML namespaces if you don't want to (and here, I don't.) - GAEUnit continues to be awesome. Android unit testing continues to be much clumsier. Yet another reason for AppEngine to do most of the tricky work.
- Optimization. Always a thorny issue. As a wise programmer once taught me:
- The first rule of optimization: don't do it.
- The second rule of optimization (For Experts Only): don't do it yet.
The android SDK comes with three excellent documents - Designing for Performance, Designing for Responsiveness, Designing for Seamlessness - that describe best practices. On the one hand, I'm not following them as closely as I could; the ExpandableListView at the heart of my UI is filled with homegrown and relatively expensive ViewEntry objects, rather than arrays as they suggest. On the other, it makes the code a lot easier to read, and ... don't optimize yet. Thus far the app seems fast and responsive enough. Says me. We'll see what others think. - The first rule of optimization: don't do it.
- HTTP GETs and POSTs. In case you'd like an example of how to do that fairly efficiently from Android, here ya go:
public static HttpResponse DoHttpPost(String target, ArrayListparams)
throws IOException
{
try {
HttpPost httpost = new HttpPost(target);
UrlEncodedFormEntity entity = new UrlEncodedFormEntity
(params, HTTP.DEFAULT_CONTENT_CHARSET);
httpost.setEntity(entity);
//configure our request
HttpParams my_httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(my_httpParams, CONNECT_TIMEOUT);
HttpConnectionParams.setSoTimeout(my_httpParams, SOCKET_TIMEOUT);
HttpClient httpclient = new DefaultHttpClient(my_httpParams); //get http client with given params
httpclient.getParams().setParameter("http.useragent", Settings.AppName);
//upload set, let's try it
Log.i("com.rezendi.wtw.Util", "Preparing to post "+params+" to "+target);
HttpResponse response = httpclient.execute(httpost);
Log.i("com.rezendi.wtw.Util", "Sent POST, got " + response.getStatusLine());
//clear out the form data
entity.consumeContent();
return response;
}
catch (IOException ex)
{
Log.e("com.rezendi.wtw.Util", "Error posting "+ params +" to "+target, ex);
throw (ex);
}
}
public static String Encode(String toEncode) throws UnsupportedEncodingException
{
return URLEncoder.encode(toEncode, "UTF-8");
}
public static String DoHttpGet(String queryUri) throws IOException
{
HttpParams my_httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(my_httpParams, Util.CONNECT_TIMEOUT);
HttpConnectionParams.setSoTimeout(my_httpParams, Util.SOCKET_TIMEOUT);
HttpClient httpclient = new DefaultHttpClient(my_httpParams); //get http client with given params
HttpGet httpget = new HttpGet(queryUri);
try {
Log.i("com.rezendi.wtw.Util","Opening HTTP GET connection to "+queryUri);
HttpResponse response = httpclient.execute(httpget);
InputStream is = response.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is), BUFFER_SIZE);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line);
}
is.close();
response.getEntity().consumeContent();
String results = sb.toString();
return results;
}
catch (IOException ex) {
Log.e("com.rezendi.wtw.Util", "Error GETting from "+queryUri, ex);
throw (ex);
}
}I recommend that you put these in one central Util class, as I did. Mind you, these are static methods on Util, and not (yet) synchronized ... meaning not thread-safe. At the moment I don't think that's an issue; all my HTTP connections are very-low-bandwidth and resolve in seconds, so I can't imagine two such calls realistically colliding unless the phone user has very fast fingers indeed. But famous last words, right? Live by the thread, die by the thread.
- In both the Android and AppEngine cases, I started off by writing the tutorials that come with the SDK, and expanded the app from there. In both cases I'm not sure this was such a good idea. I dunno, maybe it's just my OO background, but if I'm writing a Notepad application, I'd like to have an object model with a Note object. But maybe that's not really the Android way. Performance, responsiveness, seamlessness, and all that.
I did like the way they calved all direct DB access off to a DbHelper class, and I've expanded that. Here's a hint; have all your DbHelpers inherit from a common ancestor. Yeah, I know, favour composition over inheritance, but still, here it will save you a lot of time, and give you a central repository for database creation scripts and such. Also, I'm still not exactly sure when you're supposed to close your database objects. In "onPause()"? Immediately after accessing them? Don't bother and let the Activity superclass handle it? It seems unclear.
- I'm not sure of the best way to handle error handling in Android. I like to have a common superclass for all my UI objects and inherit error handling from it, but that ain't gonna happen here, as my Activities inherit from various different places on Android's Activity tree. I guess I'll build some sort of ErrorHandler and pass exceptions to it where I think they're most likely to occur? Kind of annoying that there's no single piece of code you can write to catch all exceptions before they hit Android's own error-handling, but I guess that's the price you pay for their heavily compository app system.
- Moving back to AppEngine: the memcache service is awesome. Speeds up performance and cuts down on resource consumption immensely. However, for your own good, create a "ClearCache" web handler, for debug purposes. I've twice now spent fifteen minutes wondering why a problem wasn't fixed before realizing that the erroneous data was still in the cache and had not been replaced.
- Python's HTMLParser is very useful. However, it seems to have weird problems with ampersands. I think I've now end-run around those problems, in a clumsy kludgy way, but I fear they too may yet crop up again.
- This project has included just enough JavaScript (maybe 100 lines) that I wonder if I should have used JQuery, and just little enough that I conclude that I probably shouldn't. Next time, maybe.
- I do believe that's all for now. See all y'all in beta, if not before.
Labels: Android, GET, HTTP, POST
Subscribe to Posts [Atom]