Monday, November 30, 2009
How to store images larger than 1 megabyte in Google App Engine
Over the summer, Google App Engine raised its limits for web requests and responses from 1MB to 10MB, but kept the maximum size of any single database element at 1MB. If you try to exceed this, you'll get a MemoryError. You can find a fair amount of grief and woe and gnashing of teeth and wearing of sackcloth and ashes about this online.
Which is kind of surprising, because it's not that hard to break files up into chunks and store those chunks in the database separately. Here's what I did today for my current project, which stores data - including photos - uploaded from smartphones:
First, we have to receive the uploaded image. Our uploads are two-phase - first data, then a photo - for various reasons. The data upload includes the image's file name; the photo upload is a basic form/multipart POST with exactly one argument (the filename) and its value (the file).
So, in "main.py":
and in "ec.py":
Pretty basic stuff: we chop the image up at each 1,000,000-byte mark, and put each chunk into its own ImageChunk DB object.
Then, when we need to retrieve the image, in 'main.py':
and in 'ec.py':
Since db.Blob is a subtype of str, that's all you have to do. I don't understand why some people are so upset about this: it's mildly annoying that I had to write the above, but hardly crippling. At least with JPEGs, which is what we use. (But I don't see why any other file type would be more difficult; they're ultimately all just a bunch of bytes). Could hardly be easier ... well, until App Engine rolls out their large file service.
(eta, Dec 14: which came out today! Meaning you can now disregard all the above and just use the new Blobstore instead.)
(eta, Dec 16: mmm, maybe not. Looked at the Blobstore in detail today, and it's really best suited for browser projects, not app or web-service stuff. The API for the blobs is very limited, and you can only access them via one-time-only URLs that App Engine puts in your HTML. You could scrape that, granted, but that's a pain in the ass, no less inelegant than the image-chunking solution above. It's experimental and subject to change, too. I think I'll hold out until its API improves.)
Which is kind of surprising, because it's not that hard to break files up into chunks and store those chunks in the database separately. Here's what I did today for my current project, which stores data - including photos - uploaded from smartphones:
First, we have to receive the uploaded image. Our uploads are two-phase - first data, then a photo - for various reasons. The data upload includes the image's file name; the photo upload is a basic form/multipart POST with exactly one argument (the filename) and its value (the file).
So, in "main.py":
class SaveImage(webapp.RequestHandler):
def post(self):
entryHandler=ec.EntryHandler()
for arg in self.request.arguments():
file = self.request.get(arg)
response = entryHandler.saveImage(arg,file)
self.response.out.write(response)
and in "ec.py":
class ImageChunk(db.Model):
entryRef = db.ReferenceProperty(Entry)
chunkIndex = db.IntegerProperty()
chunk = db.BlobProperty()
class EntryHandler:
def saveImage(self, fileName, file):
results = Entry.all().filter("photoPath =", fileName).fetch(1)
if len(results)==0:
logging.warning("Error - could not find the entry associated with image name "+fileName)
return "Failed"
else:
MaxBTSize=1000000
entry = results[0]
marker=0
chunks=[]
while marker*MaxBTSize<len(file):
if MaxBTSize*(marker+1)>len(file):
chunk = ImageChunk(entryRef=entry, chunkIndex=marker, chunk=db.Blob(file[MaxBTSize*marker:]))
else:
chunk = ImageChunk(entryRef=entry, chunkIndex=marker, chunk=db.Blob(file[MaxBTSize*marker:MaxBTSize*(marker+1)]))
chunk.put()
marker+=1
logging.info("Successfully received image "+fileName)
return "Successfully received image "+fileName
Pretty basic stuff: we chop the image up at each 1,000,000-byte mark, and put each chunk into its own ImageChunk DB object.
Then, when we need to retrieve the image, in 'main.py':
class ShowImageWithKey(webapp.RequestHandler):
def get(self):
key = self.request.get('entryKey')
entryHandler = ec.EntryHandler()
image = entryHandler.getImageByEntryKey(key)
if image is not None:
self.response.headers['Content-Type'] = 'image/jpeg'
self.response.out.write(image)
and in 'ec.py':
def getImageByEntryKey(self, key):
chunks = db.GqlQuery("SELECT * FROM ImageChunk WHERE entryRef = :1 ORDER BY chunkIndex", key).fetch(100)
if len(chunks)==0:
return None
image=""
for chunkRow in chunks:
image+=chunkRow.chunk
return image
Since db.Blob is a subtype of str, that's all you have to do. I don't understand why some people are so upset about this: it's mildly annoying that I had to write the above, but hardly crippling. At least with JPEGs, which is what we use. (But I don't see why any other file type would be more difficult; they're ultimately all just a bunch of bytes). Could hardly be easier ... well, until App Engine rolls out their large file service.
(eta, Dec 14: which came out today! Meaning you can now disregard all the above and just use the new Blobstore instead.)
(eta, Dec 16: mmm, maybe not. Looked at the Blobstore in detail today, and it's really best suited for browser projects, not app or web-service stuff. The API for the blobs is very limited, and you can only access them via one-time-only URLs that App Engine puts in your HTML. You could scrape that, granted, but that's a pain in the ass, no less inelegant than the image-chunking solution above. It's experimental and subject to change, too. I think I'll hold out until its API improves.)
Labels: AppEngine, BigTable, chunking, chunks, Images, JPEG, JPG, limit, MemoryError, python, size
Wednesday, November 4, 2009
to infinity, and beyond!
I am pleased to report that my pet-project iPhone app, iTravelFree, has passed the stern inspection of Apple's App Store and is now available for download worldwide. For app links, a screenshot-laden tutorial, and help and FAQ files, see here: www.wetravelright.com.
(Yeah, crappy URL, I know, but all the good ones were taken.)
Since this is my tech blog let me wax about its architecture a bit. The iPhone app is pretty straightforward: basically, it's a bunch of TableViewControllers, many of which include WebViews, along with a MapViewController, all pointing to a bunch of CoreData records. Nothing extraordinarily fancy by any means.
The server side is more interesting: it's a Google App Engine service, written in Python, that fetches, caches, and parses Wikitravel pages for the app. This gives me a single point of access to the data flow, lets me do things like convert addresses to lat/long location, cuts down on bandwidth for both Wikitravel (thanks to the caching) and the phone app (thanks to the parsing and stripping out of extraneous info.)
The general architecture - phone app plus App Engine service - is actually really powerful and easy to work with. Basically, it's a distributed version of the classic Model-View-Controller architecture, where the phone is the view, the App Engine service is the controller, and whatever data you're accessing is the model. This lets you do all the heavy-lifting computation on the server side, which is where it belongs, and keep the phone (and its puny processor) focused almost purely on the UI.
I do have some reservations about the BigTable data store that App Engine uses, but they don't apply to projects like this, with relatively simple storage requirements and no data mining.
I wrote it in, hrmm, about six weeks all told, starting in July. (Obviously it's been much more than six weeks since then, but I had full-time work starting August so could only work on this in fits and spurts on the side.)
Anyway - the app is in pretty good shape, but there's more work to be done on the server side, so it's still basically in beta test. Take a look, download it, play around, and let me know what you think -
(Yeah, crappy URL, I know, but all the good ones were taken.)
Since this is my tech blog let me wax about its architecture a bit. The iPhone app is pretty straightforward: basically, it's a bunch of TableViewControllers, many of which include WebViews, along with a MapViewController, all pointing to a bunch of CoreData records. Nothing extraordinarily fancy by any means.
The server side is more interesting: it's a Google App Engine service, written in Python, that fetches, caches, and parses Wikitravel pages for the app. This gives me a single point of access to the data flow, lets me do things like convert addresses to lat/long location, cuts down on bandwidth for both Wikitravel (thanks to the caching) and the phone app (thanks to the parsing and stripping out of extraneous info.)
The general architecture - phone app plus App Engine service - is actually really powerful and easy to work with. Basically, it's a distributed version of the classic Model-View-Controller architecture, where the phone is the view, the App Engine service is the controller, and whatever data you're accessing is the model. This lets you do all the heavy-lifting computation on the server side, which is where it belongs, and keep the phone (and its puny processor) focused almost purely on the UI.
I do have some reservations about the BigTable data store that App Engine uses, but they don't apply to projects like this, with relatively simple storage requirements and no data mining.
I wrote it in, hrmm, about six weeks all told, starting in July. (Obviously it's been much more than six weeks since then, but I had full-time work starting August so could only work on this in fits and spurts on the side.)
Anyway - the app is in pretty good shape, but there's more work to be done on the server side, so it's still basically in beta test. Take a look, download it, play around, and let me know what you think -
Labels: AppEngine, Apple, AppStore, BigTable, iPhone, iTravel, iTravelFree, python, Wikitravel
Sunday, June 7, 2009
GQL blues, or, the bloom is off the rose
So I've been doing lots of little stuff of late. I wrote my own Preferences edit screen, only to find that Android comes with an automagic one (sigh, although, I also wish this had been more obvious in the documentation.) I explored most of the mysteries of ExpandableListView, which is a pain to use, but would be an even bigger pain not to use.
And I've built the basic spine of my application - an Android app talking to an AppEngine service, which takes a few well-specified requests and feeds back a few tightly structured (and low-bandwidth) responses. The smartphone app never talks to WikiTravel directly; instead it goes through AppEngine, which can a) do all the heavy lifting, b) log and track what's going on, c) cache data more intelligently, d) provide another layer of control and indirection.
It's actually working pretty well; I'm updating WikiTravel successfully. (Well, from the emulator. I'm going to buy a real phone next week, and I expect to unearth more problems with the app when I do...) The UI is sinfully ugly, and right now it can only show you the well-formatted kind of WikiTravel listings to update, and there are a bunch of niggling things to fix and hardcoded strings to move to R.string and kludges to klean up, but by and large I'm pretty pleased with the progress.
However. I have found the first thing about the Android-AppEngine stack about which I have serious reservations. I'm talking about AppEngine's "GQL" data storage.
Now, on the one hand, this object database is quite cool. You can create, save, and write simple SQL-like queries for objects very easily. It reminds me, pardon me while I date myself, of the good old days of working with GemStone for Smalltalk, a wonderfully elegant (but ahead of its time, and hence cripplingly slow) object database for the world's purest OO language.
But if memory serves you could do a lot more with Gemstone than GQL. Especially when it comes to queries. Now, I'm biased from years of experience with SQL databases - I already know how to make them do what I want, whereas with GQL I have to work it out from a fairly thin foundation - but even so, there is much to be desired.
For instance: you can only have one inequality operator (eg "height < 5") in a query. Which means that "height > 3 AND height < 5" is actually impossible to perform. But wait, there's more; if there's an order by clause as well, it must refer to the field that uses the inequality operator. So "height < 5 ORDER BY emailAddress" is similarly impossible. Oh yes, and fetching more than 1000 rows at a time is equally impossible. As is counting all the rows that match a given WHERE clause. The official workaround for that last is to write a Sharded Counter, which, I think you'll agree, is orders of magnitude more complex than writing "SELECT COUNT(*)".
In general, getting around these restrictions means restructuring your object model heavily ... if you can get around them at all. OK, you denormalize databases for performance, and SQL has its quirks and blind spots too, but jeez, this is way past that: it's more like getting a car and being told "Listen, the steering wheel only turns right, so you have to make three right turns every time you want to go left."
I'm sure they have their reasons. Scaling being one. And for an app like mine, where the data-storage requirements are pretty simple and straightforward, this all works fine. But if I was building something seriously data-intensive? I would think not just twice but thrice before using AppEngine, at least until it supports a more fully-featured datastore.
And I've built the basic spine of my application - an Android app talking to an AppEngine service, which takes a few well-specified requests and feeds back a few tightly structured (and low-bandwidth) responses. The smartphone app never talks to WikiTravel directly; instead it goes through AppEngine, which can a) do all the heavy lifting, b) log and track what's going on, c) cache data more intelligently, d) provide another layer of control and indirection.
It's actually working pretty well; I'm updating WikiTravel successfully. (Well, from the emulator. I'm going to buy a real phone next week, and I expect to unearth more problems with the app when I do...) The UI is sinfully ugly, and right now it can only show you the well-formatted kind of WikiTravel listings to update, and there are a bunch of niggling things to fix and hardcoded strings to move to R.string and kludges to klean up, but by and large I'm pretty pleased with the progress.
However. I have found the first thing about the Android-AppEngine stack about which I have serious reservations. I'm talking about AppEngine's "GQL" data storage.
Now, on the one hand, this object database is quite cool. You can create, save, and write simple SQL-like queries for objects very easily. It reminds me, pardon me while I date myself, of the good old days of working with GemStone for Smalltalk, a wonderfully elegant (but ahead of its time, and hence cripplingly slow) object database for the world's purest OO language.
But if memory serves you could do a lot more with Gemstone than GQL. Especially when it comes to queries. Now, I'm biased from years of experience with SQL databases - I already know how to make them do what I want, whereas with GQL I have to work it out from a fairly thin foundation - but even so, there is much to be desired.
For instance: you can only have one inequality operator (eg "height < 5") in a query. Which means that "height > 3 AND height < 5" is actually impossible to perform. But wait, there's more; if there's an order by clause as well, it must refer to the field that uses the inequality operator. So "height < 5 ORDER BY emailAddress" is similarly impossible. Oh yes, and fetching more than 1000 rows at a time is equally impossible. As is counting all the rows that match a given WHERE clause. The official workaround for that last is to write a Sharded Counter, which, I think you'll agree, is orders of magnitude more complex than writing "SELECT COUNT(*)".
In general, getting around these restrictions means restructuring your object model heavily ... if you can get around them at all. OK, you denormalize databases for performance, and SQL has its quirks and blind spots too, but jeez, this is way past that: it's more like getting a car and being told "Listen, the steering wheel only turns right, so you have to make three right turns every time you want to go left."
I'm sure they have their reasons. Scaling being one. And for an app like mine, where the data-storage requirements are pretty simple and straightforward, this all works fine. But if I was building something seriously data-intensive? I would think not just twice but thrice before using AppEngine, at least until it supports a more fully-featured datastore.
Monday, June 1, 2009
zipme, baby
So I've finished the first crude version of my middleware and uploaded it to AppEngine. Looks like I'm going to have to upload the indexes by hand, though - I ran it through its paces before uploading it, but got a NeedIndexError: no matching index found when I tried to run it on appspot.com. Oh well. No biggie. I'll do that tomorrow and then go back to my Android app.
But mostly I wanted to tell you about a fun little app called zipme. AppEngine doesn't let you directly examine the source of the files you've uploaded. However, someone named "manatlan" wrote "zipme", a single python file that you add to your root directory so that you can subsequently download the entirety of your source code, zipped, from AppEngine. See here. (It's configured so you have to be logged in as admin, in case you don't wanna show your source to the world...)
eta: spoke too soon - the indexes are now up n' running. However, JavaScript form handling is not. Well, this is why you deploy early and deploy often, so that you don't get bit by it at the last minute. Goin' on a bug hunt, brb...
etaa: in case you're curious, I realized after five minutes that the culprit was neither AppEngine deployment nor my code: it was the NoScript in my browser.
But mostly I wanted to tell you about a fun little app called zipme. AppEngine doesn't let you directly examine the source of the files you've uploaded. However, someone named "manatlan" wrote "zipme", a single python file that you add to your root directory so that you can subsequently download the entirety of your source code, zipped, from AppEngine. See here. (It's configured so you have to be logged in as admin, in case you don't wanna show your source to the world...)
eta: spoke too soon - the indexes are now up n' running. However, JavaScript form handling is not. Well, this is why you deploy early and deploy often, so that you don't get bit by it at the last minute. Goin' on a bug hunt, brb...
etaa: in case you're curious, I realized after five minutes that the culprit was neither AppEngine deployment nor my code: it was the NoScript in my browser.
Labels: AppEngine, python, zipme
Saturday, May 30, 2009
a satisfied customer
I strongly endorse (at least thus far) the GAEUnit Google AppEngine unit-testing framework, available (for free) here, and insanely easy to use.
Also, don't be a total dummkopf like me and name your initial test file "unittest.py". Insert headdesk sounds here. Fortunately I realized the problem after a mere five minutes of staring at bewildering error messages.
Also, don't be a total dummkopf like me and name your initial test file "unittest.py". Insert headdesk sounds here. Fortunately I realized the problem after a mere five minutes of staring at bewildering error messages.
Labels: AppEngine, gaeunit, python, testing
Wednesday, May 27, 2009
On cloud computing nine. Well, maybe three.
I've finished the bare-bones functionality of the Android app I'm working on, and have gone back to the AppEngine middleware. The purpose of the app, in case you're curious, is to make it easy to update WikiTravel from your phone, complete with location data and/or a picture.
AppEngine is remarkably easy to work with. And as my friend Martin pointed out, it's now available in both Java and Python (although the Java is still in "Early Look" status.) I'm working in Python for the sake of variety, and also because the advantages of Java in this context are not immediately obvious.
Anyway, I don't have as much Python experience as Java, but that hardly matters, because it's all very straightforward. I'm having a permissions problem getting the location of an IFrame, which means I may have to make things a little more annoying for the user than I'd like, but that's an XSS browser-security issue not a development issue.
The basics of a web application - getting data from the database and request, displaying it to the page, and saving it as and when needed - are all perfectly straightforward; so much so that I'm not even going to bother posting any code here, for once, because none of it seems particularly interesting. Which is a good thing. It means can focus on what you want to do, unlike the bad old days, where you spent a hefty fraction of your time worrying about how you're going to do it.
(I'm sure I'll hit such a wall at some point, and fear not, when I do I will whine about it at logorrheic length.)
AppEngine also gives you lots of freebies. Sending emails easily, for once. Scalability and data integrity, for two. Goodbye, J2EE deployment descriptors; that heavy lifting now happens pretty much behind the scenes, although you do have to group objects affected by single database transactions together in advance.
You also get automatic seamless user handling, so long as you use Google Accounts as your userbase. Caveat; it's easy to link to a login page, but I haven't quite worked out how to integrate a login form into your own pages. Even so, this is pretty brilliant. It means your site comes with all the user headaches - login, logout, password reminder, sending them emails, etc. - pre-handled, saving you time and grief. It also means that if you use it, which I am for the sake of convenience, you lock yourself even further into Google's infrastructure, and expand the tentacular remit of Google Accounts. Good thing they're not evil, eh?
AppEngine is remarkably easy to work with. And as my friend Martin pointed out, it's now available in both Java and Python (although the Java is still in "Early Look" status.) I'm working in Python for the sake of variety, and also because the advantages of Java in this context are not immediately obvious.
Anyway, I don't have as much Python experience as Java, but that hardly matters, because it's all very straightforward. I'm having a permissions problem getting the location of an IFrame, which means I may have to make things a little more annoying for the user than I'd like, but that's an XSS browser-security issue not a development issue.
The basics of a web application - getting data from the database and request, displaying it to the page, and saving it as and when needed - are all perfectly straightforward; so much so that I'm not even going to bother posting any code here, for once, because none of it seems particularly interesting. Which is a good thing. It means can focus on what you want to do, unlike the bad old days, where you spent a hefty fraction of your time worrying about how you're going to do it.
(I'm sure I'll hit such a wall at some point, and fear not, when I do I will whine about it at logorrheic length.)
AppEngine also gives you lots of freebies. Sending emails easily, for once. Scalability and data integrity, for two. Goodbye, J2EE deployment descriptors; that heavy lifting now happens pretty much behind the scenes, although you do have to group objects affected by single database transactions together in advance.
You also get automatic seamless user handling, so long as you use Google Accounts as your userbase. Caveat; it's easy to link to a login page, but I haven't quite worked out how to integrate a login form into your own pages. Even so, this is pretty brilliant. It means your site comes with all the user headaches - login, logout, password reminder, sending them emails, etc. - pre-handled, saving you time and grief. It also means that if you use it, which I am for the sake of convenience, you lock yourself even further into Google's infrastructure, and expand the tentacular remit of Google Accounts. Good thing they're not evil, eh?
Labels: AppEngine, Google, Java, python
Friday, May 22, 2009
Well, that was easy.
So I created a new Google AppEngine project in Python: a very simple one, which just takes a particular HTTP POST request, stores its values to the datastore, and displays them on-screen. Then I added a "upload" function to my Android's DbHelper class, and connected the latter to the former.
I expected the debugging to be messy and lengthy. But whaddaya know? All I had to do was add the INTERNET permission to my AndroidManifest.xml, and correct the URL that I was pointing to (I'm running the AppEngine app locally; the Android emulator has a special IP address, 10.0.2.2, to connect to its host machine) and poof, amazingly, It Just Worked.
Here's the Android code, in case anyone needs an example:
I'm not even going to bother posting the server code, as it's so simple; 70 lines of Python, and 30 lines of HTML.
You'll note at the moment I just upload a picture URL, rather than actual data, but I'm going to move to doing the latter eventually.
I expected the debugging to be messy and lengthy. But whaddaya know? All I had to do was add the INTERNET permission to my AndroidManifest.xml, and correct the URL that I was pointing to (I'm running the AppEngine app locally; the Android emulator has a special IP address, 10.0.2.2, to connect to its host machine) and poof, amazingly, It Just Worked.
Here's the Android code, in case anyone needs an example:
public boolean uploadNote(long rowId, String location, String title, String comments, String picturePath) {
try {
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter("http.useragent", Util.AppName);
HttpPost httpost = new HttpPost(Util.wtwSite); // temporarily 10.0.2.2:8080/sendUpdate
Log.i(""+this, "Preparing to post to "+Util.wtwSite);
ListparamList = new LinkedList ();
paramList.add(new BasicNameValuePair("email", Util.GetUserEmail()));
paramList.add(new BasicNameValuePair("title", title));
paramList.add(new BasicNameValuePair("location", location));
paramList.add(new BasicNameValuePair("comments", comments));
paramList.add(new BasicNameValuePair("image", picturePath));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(paramList,
HTTP.DEFAULT_CONTENT_CHARSET);
httpost.setEntity(entity);
HttpResponse response = httpclient.execute(httpost);
Log.i(""+this, "Sent POST, got " + response.getStatusLine());
entity.consumeContent();
markUploaded(rowId);
return true;
}
catch (IOException ex)
{
Log.e(""+this, "Could not upload note with row "+rowId+ " due to "+ex, ex);
return false;
}
}
I'm not even going to bother posting the server code, as it's so simple; 70 lines of Python, and 30 lines of HTML.
You'll note at the moment I just upload a picture URL, rather than actual data, but I'm going to move to doing the latter eventually.
Labels: Android, AndroidManifest, AppEngine, Java, python, upload
Thursday, May 21, 2009
From Android to AppEngine
So I've finished the first (crude) iteration of my Android app, and am now moving on to creating a web application that will act as middleware.
Why a web app? Well. My Android app basically consists of grabbing the phone's current location, adding a picture and notes if the user so desires, and uploading this data to a web site. (I'm actually thinking of hiving off this basic functionality as a "Scout API" down the line, and releasing the code for future Android novices; I'm guessing that "hey, let's have people register / comment on the locations of $things1 when they come across them!" apps out there.
But a phone has very limited resources, from CPU to storage, and an annoying UI; so rather than make the user do everything on the phone, far better just to grab the on-the-spot data and do whatever further processing is required on a web site far, far away.
So I've started playing with Google AppEngine. Thus far it looks very easy, very powerful, and integrates seamlessly with Google Accounts, so you don't have to roll your own user infrastructure (and so you get locked into theirs....) It's also currently only available in Python, but what the hell, I was getting bored of Java anyway, right? More on AppEngine in the next post.
Other things I learned about Android:
Don't tug on SurfaceView's cape
The camera API is surprisingly simple, and Android comes with a SurfaceView which, among other things, lets you easily display a real-time camera preview (if you subclass it.) My intent was to have the "take picture" and "see preview" user actions be part of the same Activity - ie, when the user selects "take picture", stop showing the preview, and instead draw that snapshot on the SurfaceView.
This did not work well. SurfaceView is a very weird artifact and a difficult one to play with. After about two hours of beating my head against the problem I wisely decided to create a "ShowPicture" Activity with an ImageView, which was much simpler, and worked much better. Moral: don't overload any given Activity. Keep 'em simple, stupid.
Also, because I subclassed SurfaceView into CameraView (as per the Camera example in the API Demos that come with the SDK) I had to have a two-level callback structure: in my Camera Activity, there was a TakePicture button, which called "takePicture()" in my CameraView, which instructed its Camera to snap a shot. Then there were three callbacks in CameraView (one to indicate success, one for the RAW, one for the JPEG) and in turn I had to callback Camera. This works fine, but is a little more convoluted than I'd like.
Be careful about where you are in the stack
Creating a new ShowPicture activity did create a new problem, in that the user navigation might be "edit note -> take picture -> see preview" and might be "edit note -> see existing picture -> take new picture". In the first case, after you take the picture, you want to open a new SeePicture Activity; in the second, you want to return to the existing one. The solution was to pass a flag in the Intent that goes to the Camera activity, indicating whether you're taking a new picture or replacing an existing one.
Always save when leaving a window
In theory, Android could kill your app at any time, to conserve resources. It will probably notify you before it does so (see the Activity Life Cycle.) So it's best to save all user-modified information in "onPause()", just on case. This incentivizes you to keep your database structures very simple.
This does create the cancellation problem as per my last post, but I think that's the lesser of two evils.
GPS is busted in the emulator
The emulator's mock location providers basically don't work. Fortunately, this basically doesn't matter, as you can roll your own if you really care.
1For the record, I hate Perl with an abiding passion, but the $ prefix remains the best way to signify "this is a variable" in prose.
Why a web app? Well. My Android app basically consists of grabbing the phone's current location, adding a picture and notes if the user so desires, and uploading this data to a web site. (I'm actually thinking of hiving off this basic functionality as a "Scout API" down the line, and releasing the code for future Android novices; I'm guessing that "hey, let's have people register / comment on the locations of $things1 when they come across them!" apps out there.
But a phone has very limited resources, from CPU to storage, and an annoying UI; so rather than make the user do everything on the phone, far better just to grab the on-the-spot data and do whatever further processing is required on a web site far, far away.
So I've started playing with Google AppEngine. Thus far it looks very easy, very powerful, and integrates seamlessly with Google Accounts, so you don't have to roll your own user infrastructure (and so you get locked into theirs....) It's also currently only available in Python, but what the hell, I was getting bored of Java anyway, right? More on AppEngine in the next post.
Other things I learned about Android:
Don't tug on SurfaceView's cape
The camera API is surprisingly simple, and Android comes with a SurfaceView which, among other things, lets you easily display a real-time camera preview (if you subclass it.) My intent was to have the "take picture" and "see preview" user actions be part of the same Activity - ie, when the user selects "take picture", stop showing the preview, and instead draw that snapshot on the SurfaceView.
This did not work well. SurfaceView is a very weird artifact and a difficult one to play with. After about two hours of beating my head against the problem I wisely decided to create a "ShowPicture" Activity with an ImageView, which was much simpler, and worked much better. Moral: don't overload any given Activity. Keep 'em simple, stupid.
Also, because I subclassed SurfaceView into CameraView (as per the Camera example in the API Demos that come with the SDK) I had to have a two-level callback structure: in my Camera Activity, there was a TakePicture button, which called "takePicture()" in my CameraView, which instructed its Camera to snap a shot. Then there were three callbacks in CameraView (one to indicate success, one for the RAW, one for the JPEG) and in turn I had to callback Camera. This works fine, but is a little more convoluted than I'd like.
Be careful about where you are in the stack
Creating a new ShowPicture activity did create a new problem, in that the user navigation might be "edit note -> take picture -> see preview" and might be "edit note -> see existing picture -> take new picture". In the first case, after you take the picture, you want to open a new SeePicture Activity; in the second, you want to return to the existing one. The solution was to pass a flag in the Intent that goes to the Camera activity, indicating whether you're taking a new picture or replacing an existing one.
Always save when leaving a window
In theory, Android could kill your app at any time, to conserve resources. It will probably notify you before it does so (see the Activity Life Cycle.) So it's best to save all user-modified information in "onPause()", just on case. This incentivizes you to keep your database structures very simple.
This does create the cancellation problem as per my last post, but I think that's the lesser of two evils.
GPS is busted in the emulator
The emulator's mock location providers basically don't work. Fortunately, this basically doesn't matter, as you can roll your own if you really care.
1For the record, I hate Perl with an abiding passion, but the $ prefix remains the best way to signify "this is a variable" in prose.
Labels: Android, AppEngine, CanvasView, Java, SurfaceView
Subscribe to Posts [Atom]