Saturday, January 7, 2012

Implementing MySQL style AUTOINCREMENT in SQLite3

Yesterday I wrote about implementing the MySQL enum datatype in SQLite3, and while that exploration turned out to be pretty simple someone tweeted a followup talking about how I needed to cover the AUTOINCREMENT feature as well.  Studying my code I realized that indeed it would be necessary.  Fortunately this came out to be very simple, much simpler than was implementing the enum datatype.

To start this off let's review what the MySQL style auto_increment does.  A typical schema might include

CREATE TABLE geog (
  id int(11) NOT NULL auto_increment,
..
);

With this table definition you add items to your database with:

INSERT INTO geog(id, ..) VALUES(NULL, ..);

Basically the idea is when you supply NULL for the value of an auto_increment field, the database picks the next available value which will be one greater than the largest existing value.  This feature is useful for auto-generating unique keys for your data.

Unfortunately SQLite3 doesn't support this syntax.  And in the code I posted yesterday I skipped over the auto_increment feature by specifying this in the schema:

CREATE TABLE geog (
  id int(11) NOT NULL ,
..
);

This does not at all implement the auto_increment feature, instead it requires that values be provided for the id column.  The code running my website instead specifies NULL so that the database will provide a value for id.  If you think about it, it's difficult to implement this from the code running the site because there's a race condition in which you could write a query to get the current maximum value in the id column, then generate a new id value, but what if there are two requests underway at the same moment attempting to add rows to this table.  The two could be retrieving the maximum id value at the same time, generate the same next value for id, then both insert entries in the database with the same id value, which would erase its value as a unique identifier.  It's much better for the database to do this for us.

Fortunately the SQLite3 documentation has an answer for us (see http://www.sqlite.org/autoinc.html) that's ready to use, but with a different syntax than MySQL's implementation.  So much for SQL being a standard language, eh?

With SQLite3 you do this in your schema:

CREATE TABLE geog (
  id INTEGER PRIMARY KEY AUTOINCREMENT, -- int(11) NOT NULL ,
..
);

It turns out that in SQLite3 there is a ROWID column on every table that actually could serve very well as a unique identifier for each row.  However adding a column as defined above aliases the ROWID column to be the one you name with the datatype INTEGER PRIMARY KEY AUTOINCREMENT.  The other thing you have to do is not declare a PRIMARY KEY in the table like so:

CREATE TABLE geog (
  id INTEGER PRIMARY KEY AUTOINCREMENT, -- int(11) NOT NULL ,
..
--   PRIMARY KEY  (`id`) --  AUTOINCREMENT,
..
);

That is, here I've commented out the PRIMARY KEY declaration, because id is already declared to be a PRIMARY KEY and SQLite3 doesn't like having two PRIMARY KEY specifications.

So, that's nice, but how do you use it?

sqlite> insert into geog(id, continent, country, state, city, url) values  (1, 'Georgia', '', '', '', '');
SQL error: foreign-key violation: geog.continent
sqlite> insert into geog(id, continent, country, state, city, url) values  (1, 'Asia', '', '', '', '');
sqlite> insert into geog(id, continent, country, state, city, url) values  (NULL, 'Asia', 'Japan', '', '', '');
sqlite> insert into geog(id, continent, country, state, city, url) values  (NULL, 'Asia', 'Japan', 'Tokyo', '', '');
sqlite> insert into geog(id, continent, country, state, city, url) values  (NULL, 'Asia', 'Japan', 'Tokyo', 'Tokyo', '');
sqlite> .dump geog
BEGIN TRANSACTION;
CREATE TABLE geog (
  id INTEGER PRIMARY KEY AUTOINCREMENT, -- int(11) NOT NULL ,
  continent text, --  enum('North America','South America','Central America','Carribean','Atlantic','Europe (West)','Europe (East)','Africa','Middle East','South Asia','East Asia','Asia','Pacific','Australia') NOT NULL ,
  country char(255) NOT NULL default '',
  state char(255) NOT NULL default '',
  city char(255) NOT NULL default '',
  url char(255) default NULL
--   PRIMARY KEY  (`id`) --  AUTOINCREMENT,
--   KEY `byContinent` (`continent`),
--   KEY `byCountry` (`country`),
--   KEY `byState` (`state`),
--   KEY `byCity` (`city`),
--   KEY `geogurl` (`url`)
);
INSERT INTO "geog" VALUES(1,'Asia','','','','');
INSERT INTO "geog" VALUES(2,'Asia','Japan','','','');
INSERT INTO "geog" VALUES(3,'Asia','Japan','Tokyo','','');
INSERT INTO "geog" VALUES(4,'Asia','Japan','Tokyo','Tokyo','');
CREATE TRIGGER ContinentTrigger BEFORE INSERT ON geog FOR EACH ROW
WHEN (SELECT COUNT(*) FROM geogContinents WHERE  continentName = new.continent) = 0 BEGIN
   SELECT RAISE(rollback, 'foreign-key violation: geog.continent');
END;
COMMIT;

The first insert command demonstrates that the trigger still works to limit the continent names to the values stored in geogContinents.  The following insert's demonstrate using NULL to autogenerate id values, and then we dump the table showing the values that the database now contains.

The behavior is exactly what the code in my site expects.



Friday, January 6, 2012

Converting a MySQL enum for use in SQLite3

I've got a database & website I want to move from using MySQL to using SQLite3.  Well, I think I want to use SQLite3.  Their document saying what sorts of uses make sense for SQLite3 are directly in line with my website, and I do want to remove some of the load off of my MySQL server so that it can have  cycles free for more important purposes.

However I've run into a couple troubles converting the schema so that it fits within SQLite3's limited SQL support.  Turns out that it doesn't support some column types and indexes.  And that the SQL produced by mysqldump contains some MySQLisms which SQLite3 just doesn't understand.

One of the details is that SQLite3 doesn't support enum's.  Sigh.  Here's one of my table definitions only slightly cleaned up from mysqldump:

DROP TABLE IF EXISTS geog;
 SET @saved_cs_client     = @@character_set_client;
 SET character_set_client = utf8;
CREATE TABLE geog (
  id int(11) NOT NULL ,
  continent  enum('North America','South America','Central America','Carribean','Atlantic','Europe (West)','Europe (East)','Africa','Middle East','South Asia','East Asia','Asia','Pacific','Australia') NOT NULL ,
  country char(255) NOT NULL default '',
  state char(255) NOT NULL default '',
  city char(255) NOT NULL default '',
  url char(255) default NULL,
  PRIMARY KEY  (`id`)  AUTOINCREMENT,
   KEY `byContinent` (`continent`),
   KEY `byCountry` (`country`),
   KEY `byState` (`state`),
   KEY `byCity` (`city`),
   KEY `geogurl` (`url`)
) ENGINE=MyISAM AUTO_INCREMENT=9643 DEFAULT CHARSET=latin1 PACK_KEYS=1;
SET character_set_client = @saved_cs_client;

This table is meant to contain a list of geographic locations and I chose to use an enum to store the continent names.  There are several things in this which SQLite3 barfs on when you run it like so:

$ sqlite3 -init geog.sql geog.db
-- Loading resources from geog.sql
SQL error near line 3: near "SET": syntax error
SQL error near line 4: near "SET": syntax error
SQL error near line 5: near "'North America'": syntax error
SQL error near line 19: near "SET": syntax error
SQLite version 3.5.9
Enter ".help" for instructions
sqlite>

Most of the SET statements simply don't work so we'll comment them out in a minute.  The thing which stuck out was the "syntax error" on North America.  Took awhile to work out what that meant, had to dig through the SQLite3 documentation to learn that their type system is somewhat, um, interesting.

They have this concept of "type affinity" which means that the stored value is only loosely associated with the type name in the SQL of the table definition.  One particular thing is that enum is not one of the recognized data types.  Which leaves you with the task of mimicing the enum column type in some other way.

Basically an enum column is a text column constrained to a specific set of values.  I like the enum syntax because it's nice and concise and declarative.  But .. pragmatically .. there are several ways to implement constraints.  For example your code surrounding the database can enforce the constraint, but then you have to make sure all accesses to the database is through that code because that's the only way to enforce the constraint.

Fortunately I found a discussion of using triggers to enforce constraints and came up with this:-

DROP TABLE IF EXISTS geog;
-- SET @saved_cs_client     = @@character_set_client;
-- SET character_set_client = utf8;
CREATE TABLE geog (
  id int(11) NOT NULL ,
  continent text, --  enum('North America','South America','Central America','Carribean','Atlantic','Europe (West)','Europe (East)','Africa','Middle East','South Asia','East Asia','Asia','Pacific','Australia') NOT NULL ,
  country char(255) NOT NULL default '',
  state char(255) NOT NULL default '',
  city char(255) NOT NULL default '',
  url char(255) default NULL,
  PRIMARY KEY  (`id`) --  AUTOINCREMENT,
--   KEY `byContinent` (`continent`),
--   KEY `byCountry` (`country`),
--   KEY `byState` (`state`),
--   KEY `byCity` (`city`),
--   KEY `geogurl` (`url`)
);  -- ENGINE=MyISAM AUTO_INCREMENT=9643 DEFAULT CHARSET=latin1 PACK_KEYS=1;
--SET character_set_client = @saved_cs_client;

DROP TABLE IF EXISTS geogContinents;
CREATE TABLE geogContinents (
  continentName text
);

CREATE TRIGGER ContinentTrigger BEFORE INSERT ON geog FOR EACH ROW
WHEN (SELECT COUNT(*) FROM geogContinents WHERE  continentName = new.continent) = 0 BEGIN
   SELECT RAISE(rollback, 'foreign-key violation: geog.continent');
END;

insert into geogContinents values ('North America');
insert into geogContinents values ('South America');
insert into geogContinents values ('Central America');
insert into geogContinents values ('Carribean');
insert into geogContinents values ('Atlantic');
insert into geogContinents values ('Europe (West)');
insert into geogContinents values ('Europe (East)');
insert into geogContinents values ('Africa');
insert into geogContinents values ('Middle East');
insert into geogContinents values ('South Asia');
insert into geogContinents values ('East Asia');
insert into geogContinents values ('Asia');
insert into geogContinents values ('Pacific');
insert into geogContinents values ('Australia');

First off, notice that I commented out a bunch of stuff because SQLite3 doesn't recognize SET or KEY statements, or AUTOINCREMENT, or some other stuff.

The other thing is to change the definition of continent to text, add a new table geogContinents to hold the continent names, and a trigger to enforce the constraint that continent names can only be those named in the geogContinents table.

Now with this table definition you can initialize the database:-

$ sqlite3 -init geog2.sql geog2.db
-- Loading resources from geog2.sql
SQLite version 3.5.9
Enter ".help" for instructions
sqlite>

Then with an initialized database see that the constraints are enforced:-

$ sqlite3 geog2.db
SQLite version 3.5.9
Enter ".help" for instructions
sqlite> .tables
geog            geogContinents
sqlite> insert into geog values (1, 'Georgia', '', '', '', '');
SQL error: foreign-key violation: geog.continent
sqlite> insert into geog values (1, 'Asia', '', '', '', '');
sqlite> 



http://old.nabble.com/enum-in-SQLite-td2223029.html




Sunday, December 11, 2011

Google's new 2-step verification process, and using it with 3rd party applications (like MarsEdit)

Did you turn on 2-step verification recently in your gmail account, and then see MarsEdit stop working, and groan "NOW WHAT"?  That's what I did recently, and fortunately there is a simple way to get 2-step verification to work with applications.  The issue is that most (all?) applications do not know how to do the 2-step verification process and instead fail to log you in.  By "application" I mean a non-browser application, like MarsEdit or Picasa, or I suppose some websites that do authentication against your Google Account might also fail to work.

Google in their infinite wisdom has published a video describing the new verification process, the issue with applications, and the procedure to resolve the problem.

Let's walk through the steps.

First you go to your Google Account Settings page.  In gmail (currently) if you click on your email address in the upper toolbar, it drops down a menu-thingy that includes the phrase "Account Settings".

On the Account Settings page there are links on the Overview tab - a) 2-step verification, b) Authorizing Applications & Sites

You can enable 2-step verification from the first of those links.  The benefit of 2-step verification is an added layer of security, where you get additionally verified through an SMS message sent to your cell phone.  I'm not entirely convinced this will always work as intended because for example what happens when you're traveling outside your country and maybe don't have roaming access with your normal cell phone to the local cell phone network?  Or what if you leave your cell phone at home, and need to authenticate while you're away?  In any case the verification, when your cell phone is handy, works well, because a message pops up on your phone right away with a number that you type into the entry box and you can go your merry way.

As noted this doesn't work for Applications like MarsEdit, and that's where the second link comes in.  Here you can revoke existing access grants but the important part for our immediate needs is the lower part of the page where there's a heading reading Application-Specific Passwords.

What you do is enter an Application Name, click the button, and it tells you a gobbledygook string that's the password for that application.  Google knows enough about the gobbledygook that it can recognize the application and all will be well.  The good news is you only have to do this once per application.

It's really pretty simple and it just works.

 

Monday, November 28, 2011

Posting nicely formatted code snippets on blogger blogs, using <pre> and the SyntaxHighlighter javascript library

In the past I've done my blogging on Drupal and enjoyed the contributed modules for generating syntax highlighting on code snippets.  Now that I'm moving to Blogger one issue was how to best present code snippets on the blog.  After some searching I've found a suitable method using a browser-side JavaScript library (Syntax Highlighter), and customizing the blogger theme with a small bit of code.

Let's first start with the problem statement.

We want "code" generally to be presented with a fixed-width font if only so we have good control over indentation on code snippets.  It would be nice to automatically highlight certain things in different colors to make it easier to read the code.  It'll also be extremely useful if code snippets automatically scroll along the horizontal axis, because code snippets are often wide.

The first step is to use the <pre> tag, and to customize your theme a little bit.  Using that tag automatically, generally, takes care of forcing a fixed-width font, so we can add one small bit of CSS to make enable scrollbars for <pre> tag content.

In the Blogger dashboard, navigate to the blog you want to enable this on, then head to the template section of that blog's dashboard.  The currently selected theme will be shown at the top, so click the Customize button.  Blogger will ask you to confirm you want to modify the template.

There's a consideration to make before you take this step.  Blogger doesn't do a good job with customized templates.  You're plopped into a screen where to edit the HTML/CSS/etc code of the template, and there's no attempt to separate out the kind of additions we're about to make so that you can easily switch to another template.  You'll have to record somewhere the modifications you make, so that if you later switch templates you can re-apply the changes to that new template.

Now that you're editing the template go into the <HEAD> section of the template and insert this bit of code:

<style>
pre {
  overflow-x: scroll;
}
</style>

What this does is tell the browser to add a horizontal scrollbar to <pre> tags.

We could stop at this point, and just recommend putting code snippets within <pre> tags.  In a moment we'll go on to add the Syntax Highlighter library to make this prettier, but let's first ponder a little problem.  The <pre> tags do nothing to hide other tags from being interpreted.  Putting HTML tags within <pre> tags may cause those tags to be interpreted by the browser and it's safer to encode the tags to hide them from the browser.  It's not just tags, because often we'll have less-than signs ('<') and other special characters in code snippets that the browsers might try to interpret.

What's needed is a tool to encode a block of text using HTML entities (e.g. &lt; is the HTML entity for the less-than character).  One such tool is:  http://accessify.com/tools-and-wizards/developer-tools/quick-escape/.  If you're hip to Node.js you could also take a look at this:  https://github.com/pvorb/node-esc.  There's a whole field of knowledge about this which you can yahoogle with "escape HTML".  For now it might be enough to manually go through your code to replace less-than characters with &lt; ..?

Note that the Syntax Highlighter library we're about to install offers another method other than using <pre> tags.  That library also lets you put text inside <script> tags but I recommend against this because it doesn't degrade well.  What if the JavaScript files we're about to install become unavailable?  What if your reader's browser is configured to reject JavaScript?  What..if..what..if..?  Basically, if you just use <pre> tags with the CSS above and encode the text well, you'll have nice basic support for code snippets and no need to do anything else.  The Syntax Highlighter library gives useful additional features, but if it becomes unavailable we can just delete the <script> tags referencing the library and still have working code snippets, albeit without syntax highlighting.

Now let's look at the Syntax Highlighter library.  It's a self-contained fully featured JavaScript library for highlighting code snippets.  The JavaScript executes in the browser, so we have to place the JavaScript files somewhere and add code to the template to reference the files.

The website has some installation instructions that I found less that useful.  Here's the way I've installed it.

First, download the source from the github repository:  https://github.com/alexgorbatchev/SyntaxHighlighter

As of this writing the source repository contains two directories of importance (the other directories can be ignored):  scripts, styles

Those two directories must be uploaded to a web server so that we can add <script> and <link> tags to the template referencing the files.  The <script> and <link> tags will bring in both the JavaScript library, and the CSS used by the library, to implement syntax highlighting.

In my case I have web server space I'm renting from a hosting company, and I simply set up a sub-domain over there to house the files.  You can see the details by viewing the source code of this page - but I ask you to not use my copy of the files.  The Syntax Highlighter author also is hosting the files on his own Amazon S3 account, but he says it's costing him quite a bit of bandwidth fees from an Amazon S3 account.  I would avoid using his files because he might eventually tire of paying for this bandwidth cost.  Blogger unfortunately doesn't let us upload files to be used by the theme.

In any case once you have the files somewhere (or even if you're referencing them off Alex Gorbatchev's site) add <script> and <link> tags to the <head> section of your template.

The script tags start with this:

 <script type="text/javascript" src="<URL leading to >/scripts/shCore.js"></script> 

The bit marked "URL leading to" is the URL of the place you're storing the scripts and styles files.  For every file in the scripts directory add a similar tag.

Next reference the style-sheets using a tag like this:

 <link href="<URL leading to>/styles/shCore.css" rel="stylesheet" type="text/css" /> 

To make this clear, wherever you end up hosting the Syntax Highlighter library copy both the scripts and styles directories to that web hosting location. For every file in the styles directory add a similar tag.

There are enough JavaScript and CSS files in Syntax Highlighter that it's possible to trip across a bug in Internet Explorer where it silently fails if there are more than 30 or so style sheet files.  The problem occurs with Internet Explorer 6 and hopefully that is a decreasing problem as the old WinXP machines get retired out of existence.  In any case if you want to accommodate this problem an alternative solution is:-

<style type="text/css">
@import "<URL leading to>/styles/shCore.css";
// repeat once for each file in styles directory
</style> 

This way the impact is one style-sheet rather than the slew of them which come with Syntax Highlighter.

With these files set up we now use it by adding a class to the <pre> tags.  The class added to a <pre> tag indicate which coding language is being used so that Syntax Highlighter can do its job.  For example:

<pre class="brush:js">
function foo()
{
}
</pre>

I think the "brush:" part of the class is meant to invoke the idea of painting the syntax with the desired colors, while the part after the colon is a tag describing the brush to use.  The list of brushes are on Alex Gorbatchev's website.

 

 

 

 

Monday, November 21, 2011

Successfully hosting Drupal on nginx on Dreamhost - a Dreamhost Drupal Hosting Adventure

Among the standard performance recommendations for Drupal is to switch to the nginx web server.  Because of nginx's design it's much more performant than Apache, supposedly.  I don't know enough myself about nginx to say why it's better, other than having an understanding that nginx has an event-oriented architecture that's cleaner than Apache's.  Every so often I get on a kick of trying to use nginx based on reading some blog post or tutorial, and a couple weeks ago I came across another one that inspired me to give nginx another try (see link at bottom of this post).  And along the way I came up with a solution to the issue that's always plagued me over using nginx with Drupal.  Hopefully the result will be reliable enough so that I'll stay with nginx.

Fortunately Dreamhost has a control panel option on Dreamhost VPS's to switch between different web servers.  If you have a Dreamhost VPS you can choose between using Apache, nginx, lighthttpd OR to have no web server at all.  I'm not sure why you'd be renting a Dreamhost VPS that has no VPS but it's an option that some may want, and they'll let you do so if you want.  In any case it's easy enough to use the control panel and switch to nginx.  The "Drupal Hosting Adventure" blog post (link at the bottom) I started from assumes you have a bare Ubuntu server and takes you through all the steps of setting up nginx from scratch.  Fortunately as a Dreamhost VPS user you can skip over some of the steps.

Unfortunately just switching to nginx ins't going to give you a working Drupal setup.  So let's take this by the steps ..

Dreamhost provides us with an nginx configuration that supports multiple domains etc.  This way their control panel can still be used to host domains on a VPS configured for nginx, just as on a VPS configured for Apache.  I'm sure their intention is that we have the same control panel experience irregardless of the web server being used.

Basically there's a whole bunch of nginx configuration we don't have to do.

Dreamhost has documented their setup on the support wiki at:  http://wiki.dreamhost.com/Nginx
Their nginx setup configures an FastCGI/PHP process group for each user on the VPS that has domains its hosting.  I try to have one user per VPS to host domains, and hence my VPS has one cluster of processes.  If you have more than one user you'll have more than one cluster of processes.

This is what I mean:
[psNNNNN]$ ps auxf
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
...
root      7661  0.0  0.1  80520  4636 ?        Ss   01:10   0:00 nginx: master process /dh/nginx/bin/nginx-be -c /dh/nginx/servers/httpd-ps42313/nginx.conf -p /dh/nginx/servers/httpd-ps42313/var/
dhapache  7662  0.0  0.2  82164  6832 ?        S    01:10   0:19  \_ nginx: worker process
dhapache  7663  0.0  0.2  82164  6936 ?        S    01:10   0:18  \_ nginx: worker process
11931648  7665  0.0  0.3 138024  9796 ?        Ss   01:10   0:00 /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 26535  1.5  3.0 200316 80340 ?        S    17:22   0:55  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 28003  1.6  3.0 200528 80784 ?        S    17:23   0:56  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 28347  1.4  2.7 193284 73288 ?        S    17:23   0:48  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 28581  1.5  3.3 207232 87276 ?        S    17:24   0:53  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 28749  1.5  3.4 211672 91656 ?        S    17:24   0:52  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 29523  1.6  2.9 197508 77408 ?        R    17:24   0:54  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30294  1.5  3.1 203600 83248 ?        S    17:25   0:51  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30823  1.5  2.9 197564 77472 ?        S    17:25   0:52  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30876  1.5  2.9 198676 78564 ?        S    17:25   0:49  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30952  1.5  2.8 196496 76084 ?        S    17:25   0:50  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 30972  1.5  2.6 190212 70912 ?        S    17:25   0:51  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 31465  1.6  2.8 196136 76180 ?        S    17:26   0:52  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 31655  1.5  2.8 193820 74368 ?        S    17:26   0:51  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 31777  1.5  3.0 199040 79104 ?        S    17:26   0:49  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 31918  1.6  3.3 208132 88012 ?        S    17:26   0:51  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 32176  1.5  2.6 189148 69792 ?        S    17:27   0:48  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 32590  1.4  2.9 197028 76544 ?        S    17:27   0:46  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock
11931648 32653  1.5  3.1 202848 83236 ?        S    17:27   0:50  \_ /dh/cgi-system/php5.cgi -b /home/robogeek4/.php.sock

This is something to be aware of and you may want to enforce only one user per VPS with domains, so that it limits the number of PHP processes.

The next thing to be aware of is the nginx configuration.  They provide a configuration they think will work for most people and it looks good enough to me that you probably don't want to touch it.  However what you WILL want to do is create a per-domain configuration to handle Drupal.  Or, for that matter, any application that normally works with Apache htaccess files will need some special work to run under nginx.

This issue is endemic among web applications, that they get "clean URL's" by dint of some rewrite magic in a .htaccess file that uses mod_rewrite rules to hide the ugly URL's the application developers blessed us with.

In my opinion this is a horribly broken scenario, and that web applications should work with clean URL's at all time because user experience matters.  But there are many web applications out there that use horrid URL's and hide the ugliness behind ugly mod_rewrite rulesets.

Fortunately Drupal's ugly URL's aren't so ugly … Drupal's URL's are in the form http://example.com/index.php?q=path/to/node … which makes the rewrite rules not terribly hard to create.  But Drupal, like so many other web applications, does their URL rewriting with mod_rewrite and we have to replicate this in an nginx configuration file.

Dreamhost wants us to install per-domain nginx configuration in file(s) of this name pattern:

/home/YOURUSER/nginx/YOURDOMAIN.COM/*

Here is the configuration I'm using this minute.  It's adapted and simplified from the Drupal Hosting Adventure blog post (link below) because Dreamhost already does for us many things and we can get away with just this:

[psNNNNN]$ cat ~/nginx/davidherron.com/drupal.conf
location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
        expires max;
        log_not_found off;
}

location / {
        try_files $uri $uri/ @drupal;
}

location @drupal {
        rewrite ^/(.*)$ /index.php?q=$1 last;
}

This is pretty straightforward.  The first block sets a long expires value in the header if its a static file.  The second block tries the requested URI to see if its a file in the file system, if it is NOT one then it executes the "@drupal" rule.  The @drupal rule is what does the rewrite to change over to Drupal's particular URL format.  And, well, that's about it.  The Dreamhost Wiki has some other rules you might think about, but this is enough to get you going.

Now let's talk about how to make this reliable, because reliability is not covered at all on the Dreamhost wiki page.  What always hung me up on nginx was that it worked great for a few hours, nice, fast, wonderful, but then a day later I'd get an email from someone saying "why does the website say 504 Bad Gateway"?  Ugh.  It happened every time I tried nginx, after awhile it would get into this mode where all it does is give this ugly error page.  Unfortunately I didn't know enough about nginx to figure it out, and worst there wasn't any obvious (to me) explanation of what to do.
This time I did some yahoogling and found that this is an endemic problem with using not only PHP under nginx, but other platforms like Ruby.  (see links below)  What the Bad Gateway error tells us is that the language platform (FastCGI/PHP in this case) has crashed.  Because nginx doesn't know how to restart FastCGI all it knows is that the service isn't responding and all it can do is throw up this ugly Bad Gateway page.

Interestingly the discussion on the dreamhost forum says Dreamhost's support knows about this, but the Dreamhost Wiki doesn't say anything about this.  The pages linked below makes it clear this is a "known problem" with known solutions, namely to "monitor" FastCGI/PHP and restart it upon crashes.

What "monitor" means is to regularly inspect whether FastCGI/PHP is still running, so that it can be restarted when needed.  This is where I can't yet provide good advice, because I haven't fully fleshed out this part of the story.

I'm working with the Dreamhost support on the reliability issue.  They know about it, but aren't documenting it on their support Wiki.  Hurm.  Anyway, they have a shell script they can install.   It can detect when FastCGI processes aren't there, and restart the FastCGI/PHP processes when needed.  Talk to Dreamhost support about this.  If FastCGI/PHP isn't running the script does this:

/etc/init.d/nginx startphp 

This is a special function in the nginx init script they provide, and essentially it starts the FastCGI process group as shown above.

Their script is working well so far - except for one day when there were FastCGI/PHP processes running, but my website was non-responsive.  After awhile of trying different things I ended up rebooting the VPS (clearing up the problem) and then later I wrote this script as a first step towards detecting that condition.

[psNNNNN]$ cat check.sh
PHPINFO=`curl -f http://SUBDOMAIN.DOMAIN.com 2>/dev/null | grep phpinfo | wc -l`
if [ "$PHPINFO" -lt 1 ]; then
 echo "NEED TO REBOOT - nginx nonfunctioning"
fi

PHPVERSION=`curl -f http://SUBDOMAIN.DOMAIN.com/phpinfo.php 2>/dev/null | grep "PHP Version"`
if [ x"$PHPVERSION" = "x" ]; then
 echo "NEED TO REBOOT - PHP nonfunctioning"
fi

What I've done is - for each VPS I rent from Dreamhost - assign a dummy domain to the VPS to make it easier to access each of my VPS's.  Basically my dummy domain is more rememberable than the "psNNNNN" name which Dreamhost assigned the VPS.  What assigning a dummy domain to the VPS does is create a little website hosted on that VPS.  That little dummy website is something we can use to check whether both nginx and FastCGI/PHP are running, which is what this script does.

The first step
curl -f http://SUBDOMAIN.DOMAIN.com
Is going to cause nginx to print a directory listing of the dummy website.  Because I have a script named "phpinfo.php" in that directory, the PHPINFO variable will end up with a count indicating whether or not "phpinfo" is in the output and telling us whether the directory listing is printed telling us whether nginx is running.

The next step
curl -f http://SUBDOMAIN.DOMAIN.com/phpinfo.php
Well, this is going to run phpinfo.php (which is just a phpinfo() function invocation) and print out the results.  All the script does is grep out some text that will be in the output, "PHP Version", and if that is present it indicates that PHP is running.

I set this up to be run using cron and have verified that if there's no output cron won't send me an email, but if there is output then cron sends me an email.  Hence I have an email-based warning that FastCGI or nginx is not running.  The script is suggesting when REBOOT is desirable and could be modified to go ahead and do the reboot.

This script might be enough monitoring to implement the required reliability to keep my website running.  I'm still testing it, and after a couple days it has yet to detect a problem.
In any case I hope this has been helpful.

http://cweagans.net/blog/2011/10/26/drupal-hosting-adventure
http://hostingfu.com/article/keeping-your-php-fastcgi-processes-alive
http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:nginx:daemontools_spawnfcgi?&#comment_56be08e074b4b3b746eaad8a86a768ce
http://discussion.dreamhost.com/archive/index.php/thread-128870.html
http://www.ruby-forum.com/topic/1058059




Sunday, October 30, 2011

Correctly posting blog posts to a Blogger blog using the BloggerAPI

Google conveniently provides a fair bit of documentation and example code to use the Blogger API and do things with blogspot.com based blogs.  What I want to do in this post is walk through using the Blogger API to post blog entries to a blogger blog.  The Blogger API is one of the standard blog API's and for example is directly supported in applications like MarsEdit (which I'm using to write this post).  You may have other scenarios driving the need to programmatically manipulate your blogger blog.  In my case I'm migrating blog posts from my old Drupal based blog sites, over to blogspot based blogs.  While Drupal is a wonderful platform I realized my time is better spent writing blog posts (creating content) than maintaining the servers required to run my blogs.

We'll be using the Java based client library to the Blogger API.  Under the covers it creates ATOM based XML requests.  Because this is totally obscured by the Java client library, I think users of the library are in the dark about what they're actually doing.  It's instructive to understand something about the ATOM XML format as well as the Blogger protocol reference page.  There are useful links below to all of this.

The outline of the process is:

  • Authenticate your program with Blogger/Google
    • Gives you an authenticated com.google.gdata.client.blogger.BloggerService object
  • For each post you want to create
    • Create a BlogEntry object, filling it with the data representing the blog post
    • call service.insert with that object
    • Catch any errors, and note that blogger limits you to creating 50 posts per day via the API

Authenticating with Blogger

Google supports three authentication methods: OAuth, AuthSub, and ClientLogin.  Unfortunately their description is obfuscatory enough that I wasn't able to work out  anything but the ClientLogin method.  I'd like to use OAuth but it links over to some generic documentation that doesn't make enough sense.

The ClientLogin method is the same as e.g. going to blogger.com and entering your user name (email address) and password for a Google Account.  They mean it to be used for standalone, single-user "installed" client applications (such as a desktop application).  When applications like MarsEdit ask you for Google Account credentials, they're using the ClientLogin method.  This method has one suboptimal aspect, it requires recording your Google Account user name and password in a file on your computer.  Obviously there's a security risk of that file leaking out and being seen by other people, so be careful.

import com.google.gdata.client.blogger.BloggerService;
...
String authorName = " -- ";
String userName   = " -- ";
String userPasswd = " -- ";
String blogId     = " -- ";
...
BloggerService service = new BloggerService("exampleCo-exampleApp-1");
service.setUserCredentials(userName, userPasswd);

The userName and userPassword are the same used for login at blogger.com.  The authorName is a human-friendly name string.

Getting the blogId for a Blogger blog

You get the blogId by logging into the blogger.com, and going to the blog dashboard.   The URL for the dashboard includes a blogId= parameter that's, well, the blogId for the given blog.   It looks like this:

http://www.blogger.com/blogger.g?blogID=____blogId___#overview

Simply copy the number out of your dashboard URL, and you're good to go.

It's also possible to write a little program to list the blogs (and their blogId's) owned by a particular google account:

package blogger;
import javax.xml.parsers.ParserConfigurationException;
import org.xml.sax.SAXException;
import java.io.IOException;
import java.net.URL;
import com.google.gdata.util.AuthenticationException;
import com.google.gdata.util.ServiceException;
import com.google.gdata.client.blogger.BloggerService;
import com.google.gdata.data.Feed;
import com.google.gdata.data.Entry;

public class ListBlogs {

    static String authorName = " -- ";
    static String userName   = " -- ";
    static String userPasswd = " -- ";

    public static void main(String[] args)
            throws ParserConfigurationException, SAXException, IOException,
            AuthenticationException, ServiceException {
        BloggerService service = new BloggerService("exampleCo-exampleApp-1");
        service.setUserCredentials(userName, userPasswd);
          // Request the feed
        final URL feedUrl = new URL("http://www.blogger.com/feeds/default/blogs");
        Feed feed = service.getFeed(feedUrl, Feed.class);
        // Print the results
        System.out.println(feed.getTitle().getPlainText());
        for (int i = 0; i < feed.getEntries().size(); i++) {
            Entry entry = feed.getEntries().get(i);
            System.out.println(entry.getId() + "\t" + entry.getTitle().getPlainText());
        }
    }
}

Run this (after downloading the libraries) and it prints a table of identifier strings and blog names.  Unfortunately the identifier string is verbose, long, and the blogId we're interested in is encoded within the string.  What we need is to change that last line of code to read this way:

    System.out.println(entry.getId().split("blog-")[1] + "\t" + entry.getTitle().getPlainText());

A class (MetaData) to help you form correct URL's for the Blogger API

Using the Blogger API means constructing correct URL's.  The feedUrl above is one example, and the full list of URL's is documented on the protocol reference page.  The one we're especially interested in for creating blog posts is:

http://www.blogger.com/feeds/blogID/posts/default

I found it helpful to create a class to contain the details.  It's meant to contain methods to construct any of the URL's used with the Blogger API, at the moment it only supports creating the URL to post new blog posts.

import java.net.MalformedURLException;
import java.net.URL;

public class MetaData {
  public static final String METAFEED_URL = "http://www.blogger.com/feeds/default/blogs";
  public static final String FEED_URI_BASE = "http://www.blogger.com/feeds";
  public static final String POSTS_FEED_URI_SUFFIX = "/posts/default";
  public static final String COMMENTS_FEED_URI_SUFFIX = "/comments/default";
  
  public static URL createPostFeedURL(String blogId) throws MalformedURLException {
    return new URL(FEED_URI_BASE +"/"+ blogId + POSTS_FEED_URI_SUFFIX);
  }
}

 

Creating a BlogEntry object

This method is used to create blog posts.  Simply call it with all the argument values given here.  It assumes the variables above have been initialized, specifically the "service" has been connected as above.

import com.google.gdata.client.blogger.BloggerService;
import com.google.gdata.data.Category;
import com.google.gdata.data.DateTime;
import com.google.gdata.data.Entry;
import com.google.gdata.data.Feed;
import com.google.gdata.data.IEntry;
import com.google.gdata.data.Person;
import com.google.gdata.data.PlainTextConstruct;
import com.google.gdata.data.blogger.BlogEntry;
import com.google.gdata.util.ServiceException;
import java.io.IOException;
import java.net.URL;
import java.util.LinkedList;
...
    BloggerService service = null; // initialize as above
    String blogId = null;          // find this as discussed earlier
    String authorName = null;      // pretty human name
    String userName = null;        // blogger account name
...
    public IEntry createPost(String title, String content,
                 LinkedList categories, Boolean isDraft,
                 DateTime published, DateTime edited) throws ServiceException, IOException {
         // Create the entry to insert
         BlogEntry entry = new BlogEntry();
         entry.setTitle(new PlainTextConstruct(title));
         entry.setContent(new PlainTextConstruct(content));
         if (published != null) {
             entry.setPublished(published);
         }
         if (edited != null) {
             entry.setEdited(edited);
         }
         entry.getAuthors().add(new Person(authorName, null, userName));
         entry.setDraft(isDraft);
         entry.setCanEdit(true);

         int count = 0;
         for (String catname : categories) {
             if (count++ > 19) continue;
             Category category = new Category();
             category.setScheme("http://www.blogger.com/atom/ns#");
             category.setTerm(catname);
             entry.getCategories().add(category);
         }

         // Ask the service to insert the new entry
         URL postUrl = MetaData.createPostFeedURL(blogId);
         return service.insert(postUrl, entry);
     }

Let's walk through this carefully.

Entry objects come in many varieties and the javadoc makes this rather obtuse and obfuscated.  In any case it appears that BlogEntry is the correct Entry class to use to represent, well, a BlogEntry.  Create one of these, and then fill in its data values using the methods it exports.

The setPublished and setEdited methods are used to set the publishing and edited dates.  If you don't supply these dates, the date gets set to "NOW".  In my case I'm exporting blog posts from an existing Drupal blog and want the post in Blogger to have the same posting date.  You'll notice the published and edited objects are GData DateTime object.  In my case I'm receiving String's in an RFC-822 date format and generate that DateTime object using the provided factory method:

DateTime publ = DateTime.parseRfc822(entryPublished); 

The isDraft method determines whether the post is a draft post, in which case the public won't see it, or a real live post.

There's a loop setting categories on the post.  This corresponds to what the blogger dashboard calls a "Label".  You'll notice the setScheme method call.  I really don't understand why the Blogger client library makes you do this.  You'll see on the protocol reference page this is the required encoding in the XML for each category term.  Calling the setScheme method sets this scheme attribute in the XML, but what puzzles me is why the Blogger API client library makes you do this, and why they don't just do this under the covers for you.

<category scheme="http://www.blogger.com/atom/ns#" term="marriage" />

Another tricky bit about setting up the categories is that line which does a "continue" if "count" is greater than 19.  What I found is that if you specify 20 or more categories Blogger throws up an error.  This line of code silently drops the excess category names.

Once you have the BlogEntry object configured correctly you call service.insert.  What happens inside insert is

  • we generate the correct URL for posting blog entries using the MetaData class
  • the BlogEntry object is serialized down to the correct XML,
  • the XML s sent (authenticated) to the postUrl

Conclusion

I hope this was useful.  The task of posting to a Blogger blog is pretty simple but Google's documentation leaves a bit to be desired.

Links:

http://code.google.com/apis/blogger/ - Blogger API home page

http://code.google.com/apis/gdata/javadoc/ - Javadoc for all of Google's Java client libraries

http://code.google.com/apis/blogger/docs/2.0/developers_guide_protocol.html - Protocol reference for the Blogger API

http://code.google.com/apis/blogger/docs/2.0/developers_guide_java.html#Authenticating - Authenticating using Java client

http://code.google.com/apis/accounts/docs/AuthForInstalledApps.html

 

Saturday, October 29, 2011

Correctly posting blog posts to a Blogger blog using the BloggerAPI

Google conveniently provides a fair bit of documentation and example code to use the Blogger API and do things with blogspot.com based blogs.  What I want to do in this post is walk through using the Blogger API to post blog entries to a blogger blog.  The Blogger API is one of the standard blog API's and for example is directly supported in applications like MarsEdit (which I'm using to write this post).  You may have other scenarios driving the need to programmatically manipulate your blogger blog.  In my case I'm migrating blog posts from my old Drupal based blog sites, over to blogspot based blogs.  While Drupal is a wonderful platform I realized my time is better spent writing blog posts (creating content) than maintaining the servers required to run my blogs.

We'll be using the Java based client library to the Blogger API.  Under the covers it creates ATOM based XML requests.  Because this is totally obscured by the Java client library, I think users of the library are in the dark about what they're actually doing.  It's instructive to understand something about the ATOM XML format as well as the Blogger protocol reference page.  There are useful links below to all of this.

The outline of the process is:

  • Authenticate your program with Blogger/Google
    • Gives you an authenticated com.google.gdata.client.blogger.BloggerService object
  • For each post you want to create
    • Create a BlogEntry object, filling it with the data representing the blog post
    • call service.insert with that object
    • Catch any errors, and note that blogger limits you to creating 50 posts per day via the API

Authenticating with Blogger

Google supports three authentication methods: OAuth, AuthSub, and ClientLogin.  Unfortunately their description is obfuscatory enough that I wasn't able to work out  anything but the ClientLogin method.  I'd like to use OAuth but it links over to some generic documentation that doesn't make enough sense.

The ClientLogin method is the same as e.g. going to blogger.com and entering your user name (email address) and password for a Google Account.  They mean it to be used for standalone, single-user "installed" client applications (such as a desktop application).  When applications like MarsEdit ask you for Google Account credentials, they're using the ClientLogin method.  This method has one suboptimal aspect, it requires recording your Google Account user name and password in a file on your computer.  Obviously there's a security risk of that file leaking out and being seen by other people, so be careful.

import com.google.gdata.client.blogger.BloggerService;
...
String authorName = " -- ";
String userName   = " -- ";
String userPasswd = " -- ";
String blogId     = " -- ";
...
BloggerService service = new BloggerService("exampleCo-exampleApp-1");
service.setUserCredentials(userName, userPasswd);

The userName and userPassword are the same used for login at blogger.com.  The authorName is a human-friendly name string.

Getting the blogId for a Blogger blog

You get the blogId by logging into the blogger.com, and going to the blog dashboard.   The URL for the dashboard includes a blogId= parameter that's, well, the blogId for the given blog.   It looks like this:

http://www.blogger.com/blogger.g?blogID=____blogId___#overview

Simply copy the number out of your dashboard URL, and you're good to go.

It's also possible to write a little program to list the blogs (and their blogId's) owned by a particular google account:

package blogger;
import javax.xml.parsers.ParserConfigurationException;
import org.xml.sax.SAXException;
import java.io.IOException;
import java.net.URL;
import com.google.gdata.util.AuthenticationException;
import com.google.gdata.util.ServiceException;
import com.google.gdata.client.blogger.BloggerService;
import com.google.gdata.data.Feed;
import com.google.gdata.data.Entry;

public class ListBlogs {

    static String authorName = " -- ";
    static String userName   = " -- ";
    static String userPasswd = " -- ";

    public static void main(String[] args)
            throws ParserConfigurationException, SAXException, IOException,
            AuthenticationException, ServiceException {
        BloggerService service = new BloggerService("exampleCo-exampleApp-1");
        service.setUserCredentials(userName, userPasswd);
          // Request the feed
        final URL feedUrl = new URL("http://www.blogger.com/feeds/default/blogs");
        Feed feed = service.getFeed(feedUrl, Feed.class);
        // Print the results
        System.out.println(feed.getTitle().getPlainText());
        for (int i = 0; i < feed.getEntries().size(); i++) {
            Entry entry = feed.getEntries().get(i);
            System.out.println(entry.getId() + "\t" + entry.getTitle().getPlainText());
        }
    }
}

Run this (after downloading the libraries) and it prints a table of identifier strings and blog names.  Unfortunately the identifier string is verbose, long, and the blogId we're interested in is encoded within the string.  What we need is to change that last line of code to read this way:

    System.out.println(entry.getId().split("blog-")[1] + "\t" + entry.getTitle().getPlainText());

A class (MetaData) to help you form correct URL's for the Blogger API

Using the Blogger API means constructing correct URL's.  The feedUrl above is one example, and the full list of URL's is documented on the protocol reference page.  The one we're especially interested in for creating blog posts is:

http://www.blogger.com/feeds/blogID/posts/default

I found it helpful to create a class to contain the details.  It's meant to contain methods to construct any of the URL's used with the Blogger API, at the moment it only supports creating the URL to post new blog posts.

import java.net.MalformedURLException;
import java.net.URL;

public class MetaData {
  public static final String METAFEED_URL = "http://www.blogger.com/feeds/default/blogs";
  public static final String FEED_URI_BASE = "http://www.blogger.com/feeds";
  public static final String POSTS_FEED_URI_SUFFIX = "/posts/default";
  public static final String COMMENTS_FEED_URI_SUFFIX = "/comments/default";
  
  public static URL createPostFeedURL(String blogId) throws MalformedURLException {
    return new URL(FEED_URI_BASE +"/"+ blogId + POSTS_FEED_URI_SUFFIX);
  }
}

 

Creating a BlogEntry object

This method is used to create blog posts.  Simply call it with all the argument values given here.  It assumes the variables above have been initialized, specifically the "service" has been connected as above.

import com.google.gdata.client.blogger.BloggerService;
import com.google.gdata.data.Category;
import com.google.gdata.data.DateTime;
import com.google.gdata.data.Entry;
import com.google.gdata.data.Feed;
import com.google.gdata.data.IEntry;
import com.google.gdata.data.Person;
import com.google.gdata.data.PlainTextConstruct;
import com.google.gdata.data.blogger.BlogEntry;
import com.google.gdata.util.ServiceException;
import java.io.IOException;
import java.net.URL;
import java.util.LinkedList;
...
    BloggerService service = null; // initialize as above
    String blogId = null;          // find this as discussed earlier
    String authorName = null;      // pretty human name
    String userName = null;        // blogger account name
...
    public IEntry createPost(String title, String content,
                 LinkedList categories, Boolean isDraft,
                 DateTime published, DateTime edited) throws ServiceException, IOException {
         // Create the entry to insert
         BlogEntry entry = new BlogEntry();
         entry.setTitle(new PlainTextConstruct(title));
         entry.setContent(new PlainTextConstruct(content));
         if (published != null) {
             entry.setPublished(published);
         }
         if (edited != null) {
             entry.setEdited(edited);
         }
         entry.getAuthors().add(new Person(authorName, null, userName));
         entry.setDraft(isDraft);
         entry.setCanEdit(true);

         int count = 0;
         for (String catname : categories) {
             if (count++ > 19) continue;
             Category category = new Category();
             category.setScheme("http://www.blogger.com/atom/ns#");
             category.setTerm(catname);
             entry.getCategories().add(category);
         }

         // Ask the service to insert the new entry
         URL postUrl = MetaData.createPostFeedURL(blogId);
         return service.insert(postUrl, entry);
     }

Let's walk through this carefully.

Entry objects come in many varieties and the javadoc makes this rather obtuse and obfuscated.  In any case it appears that BlogEntry is the correct Entry class to use to represent, well, a BlogEntry.  Create one of these, and then fill in its data values using the methods it exports.

The setPublished and setEdited methods are used to set the publishing and edited dates.  If you don't supply these dates, the date gets set to "NOW".  In my case I'm exporting blog posts from an existing Drupal blog and want the post in Blogger to have the same posting date.  You'll notice the published and edited objects are GData DateTime object.  In my case I'm receiving String's in an RFC-822 date format and generate that DateTime object using the provided factory method:

DateTime publ = DateTime.parseRfc822(entryPublished); 

The isDraft method determines whether the post is a draft post, in which case the public won't see it, or a real live post.

There's a loop setting categories on the post.  This corresponds to what the blogger dashboard calls a "Label".  You'll notice the setScheme method call.  I really don't understand why the Blogger client library makes you do this.  You'll see on the protocol reference page this is the required encoding in the XML for each category term.  Calling the setScheme method sets this scheme attribute in the XML, but what puzzles me is why the Blogger API client library makes you do this, and why they don't just do this under the covers for you.

<category scheme="http://www.blogger.com/atom/ns#" term="marriage" />

Another tricky bit about setting up the categories is that line which does a "continue" if "count" is greater than 19.  What I found is that if you specify 20 or more categories Blogger throws up an error.  This line of code silently drops the excess category names.

Once you have the BlogEntry object configured correctly you call service.insert.  What happens inside insert is

  • we generate the correct URL for posting blog entries using the MetaData class
  • the BlogEntry object is serialized down to the correct XML,
  • the XML s sent (authenticated) to the postUrl

Conclusion

I hope this was useful.  The task of posting to a Blogger blog is pretty simple but Google's documentation leaves a bit to be desired.

Links:

http://code.google.com/apis/blogger/ - Blogger API home page

http://code.google.com/apis/gdata/javadoc/ - Javadoc for all of Google's Java client libraries

http://code.google.com/apis/blogger/docs/2.0/developers_guide_protocol.html - Protocol reference for the Blogger API

http://code.google.com/apis/blogger/docs/2.0/developers_guide_java.html#Authenticating - Authenticating using Java client

http://code.google.com/apis/accounts/docs/AuthForInstalledApps.html